diff --git a/docs/assets/images/raphtory_ui_playground_docs.png b/docs/assets/images/raphtory_ui_playground_docs.png new file mode 100644 index 0000000000..c424619e28 Binary files /dev/null and b/docs/assets/images/raphtory_ui_playground_docs.png differ diff --git a/docs/user-guide/algorithms/4_view-algorithms.md b/docs/user-guide/algorithms/4_view-algorithms.md index 84bfc43e56..40ae2bbff6 100644 --- a/docs/user-guide/algorithms/4_view-algorithms.md +++ b/docs/user-guide/algorithms/4_view-algorithms.md @@ -1,6 +1,6 @@ -# Running algorithms on graph views +# Running algorithms on GraphViews -Both `graphwide` and `node centric` algorithms can be run on `graph views`. This allows us to see how results change over time, run algorithms on subsets of the layers, or remove specific nodes from the graph to see the impact this has. +Both `graphwide` and `node centric` algorithms can be run on `GraphViews`. This allows us to see how results change over time, run algorithms on subsets of the layers, or remove specific nodes from the graph to see the impact this has. To demonstrate this, the following example shows how you could track Gandalf's importance over the course of the story using rolling windows and the `PageRank` algorithm. diff --git a/docs/user-guide/export/2_dataframes.md b/docs/user-guide/export/2_dataframes.md index 3ebacbde64..fa70d78379 100644 --- a/docs/user-guide/export/2_dataframes.md +++ b/docs/user-guide/export/2_dataframes.md @@ -124,7 +124,7 @@ export the property history for each edge, split by edge layer. This is because explode the edges and view each update individually (which will then ignore the `include_property_history` flag). In the below example we first create a subgraph of the monkey interactions, selecting `ANGELE` and `FELIPE` as the -monkeys we are interested in. This isn't a required step, but helps to demonstrate the export of graph views. +monkeys we are interested in. This isn't a required step, but helps to demonstrate the export of GraphViews. Then we call `to_df()` on the subgraph edges, setting no flags. In the output you can see the property history for each interaction type (layer) between `ANGELE` and `FELIPE`. diff --git a/docs/user-guide/getting-started/1_intro.md b/docs/user-guide/getting-started/1_intro.md index 66680ebc47..a08719a588 100644 --- a/docs/user-guide/getting-started/1_intro.md +++ b/docs/user-guide/getting-started/1_intro.md @@ -113,6 +113,9 @@ client.send_graph("OBS-graph", g, overwrite=True) This will start the UI locally on the default port `1736`, you should see **Search** page by default. +!!! Note + You can also start a standalone server using the Raphtory CLI tool or Docker image. + ![UI Search page](../../assets/images/raphtory_ui_search_empty.png) You can use the **Query Builder** to select the graph you created and identify which baboons attacked each other in the last month. diff --git a/docs/user-guide/getting-started/3_cli.md b/docs/user-guide/getting-started/3_cli.md index f96e50aba9..b7600c5522 100644 --- a/docs/user-guide/getting-started/3_cli.md +++ b/docs/user-guide/getting-started/3_cli.md @@ -23,6 +23,7 @@ raphtory server --port 1736 | Command | Parameter(s) | Description | |-----------------------------|---------------------------|---------------------------------------------------------------------| | -h, --help | | Show the help message and exit | +| --work-dir | WORK_DIR | Working directory | | --cache-capacity | CACHE_CAPACITY | Cache capacity | | --cache-tti-seconds | CACHE_TTI_SECONDS | Cache time-to-idle in seconds | | --log-level | LOG_LEVEL | Log level | diff --git a/docs/user-guide/graphql/1_intro.md b/docs/user-guide/graphql/1_intro.md index f8c750b61f..24aa4a85e8 100644 --- a/docs/user-guide/graphql/1_intro.md +++ b/docs/user-guide/graphql/1_intro.md @@ -2,6 +2,6 @@ [GraphQL](https://graphql.org/) is a query language for your API, and a server-side runtime for executing queries using a type system you define for your data. Using GraphQL can help you can reduce over-fetching and under-fetching of data compared to other REST APIs. -It is possible to query Raphtory graphs in GraphQL. The GraphQL server provides an IDE available at `localhost:1736/playground` where you can write GraphQL queries. Alternatively, you can write all your GraphQL queries in Python and easily update, send and receive Raphtory graphs from the GraphQL server. +It is possible to query and modify Raphtory graphs in GraphQL. The GraphQL server provides an IDE available at `localhost:1736/playground` where you can write GraphQL queries. Alternatively, you can write all your GraphQL queries in Python and easily update, send and receive Raphtory graphs from the GraphQL server. This section will show you how to start a GraphQL server and run your own queries on your data. diff --git a/docs/user-guide/graphql/2_run-server.md b/docs/user-guide/graphql/2_run-server.md index a4e700e2ea..dca048d36d 100644 --- a/docs/user-guide/graphql/2_run-server.md +++ b/docs/user-guide/graphql/2_run-server.md @@ -23,38 +23,25 @@ g.save_to_file(working_dir + "your_graph") ``` /// -## Starting a server with .run() +## Starting a server -To run the GraphQL server with `.run()`, create a python file `run_server.py` with the following code: +You can start the raphtory GraphQL in multiple ways depending on your usecase. -/// tab | :fontawesome-brands-python: Python -```{.python notest} -from raphtory import graphql - -import argparse -parser = argparse.ArgumentParser(description="For passing the working_dir") -parser.add_argument( - "--working_dir", - type=str, - help="path for the working directory of the raphtory server", -) -args = parser.parse_args() +### Using the CLI -server = graphql.GraphServer(args.working_dir) +You can use the [Raphtory CLI](../getting-started/3_cli.md) with the `server` command by running: -server.run() +```sh +raphtory server --port 1736 ``` -/// -To run the server: +This option is the simplist and provides the most configuration options. -```bash -python run_server.py --working_dir ../your_working_dir -``` +### Start a server in Python -## Starting a server with .start() +If you have a [`GraphServer`][raphtory.graphql.GraphServer] object you can use either the [`.run()`][raphtory.graphql.GraphServer.run] or [`.start()`][raphtory.graphql.GraphServer.start] functions to start a GraphQL sever and Raphtory UI. -It is also possible to start the server in Python with `.start()`. Below is an example of how to start the server and send a Raphtory graph to the server, where `new_graph` is your Raphtory graph object. +Below is an example of how to start the server and send a Raphtory graph to the server, where `new_graph` is your Raphtory graph object. /// tab | :fontawesome-brands-python: Python ```{.python notest} diff --git a/docs/user-guide/graphql/3_writing-queries.md b/docs/user-guide/graphql/3_writing-queries.md index c1ef906cb4..809c477197 100644 --- a/docs/user-guide/graphql/3_writing-queries.md +++ b/docs/user-guide/graphql/3_writing-queries.md @@ -1,12 +1,20 @@ -# Writing Raphtory queries in GraphQL +# Making GraphQL requests + +The GraphQL API largely follows the same patterns as the Python API but has a few key differences. + +In GraphQL, there are two different types of requests: a query to search through your data or a mutation of your data. Only the top-level fields in mutation operations are allowed to cause side effects. To accommodate this, in the Raphtory API you can make queries to graphs or metagraphs but must make changes using a mutable graph, node or edge. + +This division means that the distinction between Graphs and GraphViews is less important in GraphQL and all non-mutable Graph endpoints are GraphViews while MutableGraphs are used for mutation operations. This is also true for Nodes and Edges and their respective views. Graphs can be further distinguished as either `PERSISTENT` or `EVENT` types. + +## Graphical playground When you start a GraphQL server, you can find your GraphQL UI in the browser at `localhost:1736/playground` or an alternative port if you specified one. -The schema for the queries can be found on the right hand side in a pull out toggle. +An annotated schema is available from the documentation tab in the left hand menu of the playground. -![alt text](schema.png) +![alt text](../../assets/images/raphtory_ui_playground_docs.png) -## Example Queries in GraphQL +## Query a graph Here are some example queries to get you started: @@ -26,7 +34,7 @@ query { ``` /// -## List of all the edges, with specific node properties +### List of all the edges, with specific node properties To find nodes with `age`: @@ -59,7 +67,7 @@ query { ``` /// -This will return something like this: +This will return something like: !!! Output @@ -120,9 +128,163 @@ g.node("Ben").properties.get("age") ``` /// -## Querying GraphQL in Python +### Examine the metadata of a node + +Metadata does not change over the lifetime of an object. You can request it with a query like the following: + + +/// tab | ![GraphQL](https://img.icons8.com/ios-filled/15/graphql.png) GraphQL +``` +{ + graph(path: "traffic_graph") { + nodes { + list { + name + metadata { + values { + key + value + } + } + } + } + } +} +``` +/// + +Which will return something like: + +!!! Output + ```json + { + "data": { + "graph": { + "nodes": { + "list": [ + { + "name": "ServerA", + "metadata": { + "values": [ + { + "key": "datasource", + "value": "network_traffic_edges.csv" + }, + { + "key": "server_name", + "value": "Alpha" + }, + { + "key": "hardware_type", + "value": "Blade Server" + } + ] + } + }, + { + "name": "ServerB", + "metadata": { + "values": [ + { + "key": "datasource", + "value": "network_traffic_edges.csv" + }, + { + "key": "server_name", + "value": "Beta" + }, + { + "key": "hardware_type", + "value": "Rack Server" + } + ] + } + } + ] + } + } + } + } + ``` + +### Examine the properties of a node + +Properties can change over time so it is often useful to make a query for a specific time or window. + +/// tab | ![GraphQL](https://img.icons8.com/ios-filled/15/graphql.png) GraphQL +``` +{ + graph(path: "traffic_graph") { + at(time: 1693555500000) { + nodes { + list { + name + properties { + values { + key + value + } + } + } + } + } + } +} +``` +/// + +Which will return something like: -It is possible to send GraphQL queries in Python without the in-browser IDE. This can be useful if you want to update your Raphtory graph in Python. This example shows you how to do this with the Raphtory client: +!!! Output + ```json + { + "data": { + "graph": { + "at": { + "nodes": { + "list": [ + { + "name": "ServerA", + "properties": { + "values": [] + } + }, + { + "name": "ServerB", + "properties": { + "values": [ + { + "key": "OS_version", + "value": "Red Hat 8.1" + }, + { + "key": "primary_function", + "value": "Web Server" + }, + { + "key": "uptime_days", + "value": 45 + } + ] + } + }, + { + "name": "ServerC", + "properties": { + "values": [] + } + } + ] + } + } + } + } + } + ``` + +### Querying GraphQL in Python + +You can also send GraphQL queries in Python directl using the [`.query()`][raphtory.graphql.RaphtoryClient.query] function on a `RaphtoryClient`. The following example shows you how to do this: /// tab | :fontawesome-brands-python: Python ```{.python notest} @@ -144,40 +306,16 @@ Pass your graph object string into the `client.query()` method to execute the Gr {'graph': {'created': 1729075008085, 'lastOpened': 1729075036222, 'lastUpdated': 1729075008085}} ``` +## Mutation requests -## Mutation Queries - -In GraphQL, you can write two different types of queries - a query to search through your data or a query that mutates your data. +You can also mutate your graph. This can be done both in the GraphQL IDE and in Python. -The examples in the previous section are all queries used to search through your data. However in our API, you can also mutate your graph. This can be done both in the GraphQL IDE and in Python. - -The schema in the GraphQL IDE shows how you can mutate the graph within the IDE: - -``` -type MutRoot { - plugins: MutationPlugin! - deleteGraph(path: String!): Boolean! - newGraph(path: String!, graphType: GqlGraphType!): Boolean! - moveGraph(path: String!, newPath: String!): Boolean! - copyGraph(path: String!, newPath: String!): Boolean! - - # Use GQL multipart upload to send new graphs to server - # - # Returns:: - # name of the new graph - uploadGraph(path: String!, graph: Upload!, overwrite: Boolean!): String! - - # Send graph bincode as base64 encoded string - # - # Returns:: - # path of the new graph - sendGraph(path: String!, graph: String!, overwrite: Boolean!): String! -} -``` +From GraphQL these operations are available from the [Mutation root](../../../reference/graphql/graphql_API/#mutation-mutroot) which operates on mutable objects by specified by a path. -There are additional methods to mutate the graph exclusive to Python such as sending, receiving and updating a graph, these will all be explained below. +!!! note + Some methods to mutate the graph are exclusive to Python. -## Sending a graph +### Sending a graph You can send a graph to the server and overwrite an existing graph if needed. @@ -223,7 +361,7 @@ This should return: } ``` -## Receiving graphs +### Receiving graphs You can retrieve graphs from a "path" on the server which returns a Python Raphtory graph object. @@ -234,7 +372,7 @@ g.edge("sally", "tony") ``` /// -## Creating a new graph +### Creating a new graph This is an example of how to create a new graph in the server. @@ -272,7 +410,7 @@ The returning result to confirm that a new graph has been created: } ``` -## Moving a graph +### Moving a graph It is possible to move a graph to a new path on the server. @@ -307,7 +445,7 @@ The returning GraphQL result to confirm that the graph has been moved: } ``` -## Copying a graph +### Copying a graph It is possible to copy a graph to a new path on the server. @@ -342,7 +480,7 @@ The returning GraphQL result to confirm that the graph has been copied: } ``` -## Deleting a graph +### Deleting a graph It is possible to delete a graph on the server. @@ -377,7 +515,7 @@ The returning GraphQL result to confirm that the graph has been deleted: } ``` -## Updating the graph +### Updating the graph It is possible to update the graph using the `remote_graph()` method. diff --git a/docs/user-guide/graphql/4_running-ui.md b/docs/user-guide/graphql/4_running-ui.md deleted file mode 100644 index fe434cf082..0000000000 --- a/docs/user-guide/graphql/4_running-ui.md +++ /dev/null @@ -1,17 +0,0 @@ -# Running the UI - -Raphtory allows you to easily set up a sophisticated UI to examine and analyze your data. - -To run the UI, you will first need to have the [GraphQL server](../../user-guide/graphql/2_run-server.md) running. Once the server is running, the UI will be available on the same port. - -## Search page - -The search page of the UI is used to search and filter your data. For example, you can narrow your search by date, graph name and even node properties. You can view a node's direct connections as well as its activity log and history. By double clicking on one of the results, you can navigate to the graph page where you will see a graphical representation of your data. - -![alt text](search_page.png) - -## Graph page - -The graph page of the UI is used to explore your data and graph in an interactive way. Here you will be able to see all your nodes and edges in a clear format, with the ability to delete, expand and so much more. - -![alt text](graph_page.png) diff --git a/docs/user-guide/ingestion/2_direct-updates.md b/docs/user-guide/ingestion/2_direct-updates.md index cd8698a62c..d0a628bcb5 100644 --- a/docs/user-guide/ingestion/2_direct-updates.md +++ b/docs/user-guide/ingestion/2_direct-updates.md @@ -325,7 +325,7 @@ separately or merged with other layers as required. You can see this in the example below where we add five updates between `Person 1` and `Person 2` across the layers `Friends`, `Co Workers` and `Family`. When we query the history of the `weight` property on the edge we initially get -all of the values back. However, by applying the [`layers()` graph view](../views/3_layer.md) we can return only updates +all of the values back. However, by applying the [`layers()` GraphView](../views/3_layer.md) we can return only updates from `Co Workers` and `Family`. /// tab | :fontawesome-brands-python: Python diff --git a/docs/user-guide/installation.md b/docs/user-guide/installation.md index 629bde1606..09b32f7efa 100644 --- a/docs/user-guide/installation.md +++ b/docs/user-guide/installation.md @@ -22,7 +22,6 @@ raphtory = { version = "x"} To use the library import it into your project: - /// tab | :fontawesome-brands-python: Python ``` python import raphtory as rp @@ -34,3 +33,33 @@ import raphtory as rp use raphtory::prelude::*; ``` /// + +## Docker image + +Both the Python and Rust packages are available as official Docker images from the [Pometry Docker Hub](https://hub.docker.com/r/pometry/raphtory) page. + +To download these using the docker CLI run: + +/// tab | :fontawesome-brands-python: Python +``` bash +docker pull pometry/raphtory:latest-python +``` +/// + +/// tab | :fontawesome-brands-rust: Rust +``` shell +docker pull pometry/raphtory +``` +/// + +Running either container will start a Raphtory server by default, if this is all you need then the Rust image is sufficient. + +However, the Python image contains the Raphtory Python package and all the required dependencies. You should use this image if you want to develop using the Python APIs in a containerised environment. + +You can run a Raphtory container with the following Docker command: + +```docker +docker run --rm -p 1736:1736 -v "$(pwd):/home/raphtory_server" pometry/raphtory:latest-python +``` + +For more information about running and configuring containers see the [Docker documentation](https://docs.docker.com/). diff --git a/docs/user-guide/troubleshooting.md b/docs/user-guide/troubleshooting.md new file mode 100644 index 0000000000..f689971fc2 --- /dev/null +++ b/docs/user-guide/troubleshooting.md @@ -0,0 +1,11 @@ +# Troubleshooting + +This page covers common errors and misconfigurations in Raphtory. + +## Specifying time measurements + +Internally all times in Raphtory are represented as milliseconds using unix epochs. When ingesting data you will need to convert your raw data into the appropriate format. Similarly queries made using the API should use timestamps relative to the unix epoch in milliseconds. + +## Docker graph storage + +When saving a graph to disk in the official Docker container, the default location is `/home/raphtory_server`. This is where the Raphtory server will look for graphs unless you specify an alternative working directory. When saving a file or sending a graph to the server you can always specify a custom path. diff --git a/docs/user-guide/views/1_intro.md b/docs/user-guide/views/1_intro.md index d61dfe2906..16f0ac8989 100644 --- a/docs/user-guide/views/1_intro.md +++ b/docs/user-guide/views/1_intro.md @@ -1,8 +1,8 @@ # Introduction and dataset -Many operations are executed on the whole graph, including the full history. In this section we will look at applying `Graph Views` which provide a way to look at a subset of this data without having to re-ingest it. +Many operations are executed on the whole graph, including the full history. In this section we will look at applying `GraphViews` which provide a way to look at a subset of this data without having to re-ingest it. -Raphtory can maintain hundreds of thousands of `Graph Views` in parallel and allows chaining view functions together to create as specific a filter as is required for your use case. A unified API means that all functions that can be called on a graph, node or edge can also be applied to this subset. +Raphtory can maintain hundreds of thousands of `GraphViews` in parallel and allows chaining view functions together to create as specific a filter as is required for your use case. A unified API means that all functions that can be called on a graph, node or edge can also be applied to this subset. !!! info diff --git a/docs/user-guide/views/2_time.md b/docs/user-guide/views/2_time.md index da464e3e95..4489289f30 100644 --- a/docs/user-guide/views/2_time.md +++ b/docs/user-guide/views/2_time.md @@ -1,7 +1,7 @@ # Querying the graph over time Raphtory provides six functions: `before()`, `at()`, `after()`, `window()`, `expand()` and `rolling()` for traveling through time and viewing a graph as it was at a specific point, or between two points (applying a time window). -All of these functions can be called on a `graph`, `node`, or `edge`, returning an equivalent `Graph View`, `Node View` or `Edge View` which have all the same functions as its unfiltered counterpart. This means that if you write a function which takes a Raphtory entity, regardless of which filters have been applied. +All of these functions can be called on a `graph`, `node`, or `edge`, returning an equivalent `GraphView`, `NodeView` or `EdgeView` which have all the same functions as its unfiltered counterpart. This means that if you write a function which takes a Raphtory entity, regardless of which filters have been applied. ## Before, At and After @@ -129,15 +129,16 @@ assert str(f"Window start: {w.start_date_time}, First update: {w.earliest_date_t ``` ## Traversing the graph with views + There are important differences when applying views depending on which object you call them on because of how filters propagate as you traverse the graph. - -As a general rule, when you call any function which returns another entity, on a `Graph View`, `Node View` or `Edge View`, the view's filters will be passed onto the entities it returns. For example, if you call `before()` on a graph and then call `node()`, this will return a `Node View` filtered to the time passed to the graph. + +As a general rule, when you call any function which returns another entity, on a `GraphView`, `NodeView` or `EdgeView`, the view's filters will be passed onto the entities it returns. For example, if you call `before()` on a graph and then call `node()`, this will return a `NodeView` filtered to the time passed to the graph. However, if this was always the case it would be limiting if you later wanted to explore outside of these bounds. To allow for both global bounds and moving bounds, if a filter is applied onto the graph, all entities extracted always have this filter applied. However, if a filter is applied to either a `node` or an `edge`, once you have traversed to a new neighbouring `node` this filter is removed. -As an example of this, below we look at LOME's one hop neighbours before the 20th of June and their neighbours (LOME's two hop neighbours) after the 25th of June. +As an example of this, below we look at LOME's one hop neighbours before the 20th of June and their neighbours (LOME's two hop neighbours) after the 25th of June. First we show calling `before()` on the `graph`. This works for the one hop neighbours, but when `after()` is applied the graph is empty as there is no overlap in dates between the two filters. Next we show calling `before()` on the `node` instead. In this case, once the neighbours have been reached the original filter is removed which allows `after()` to work as desired. diff --git a/mkdocs.yml b/mkdocs.yml index ebdf6fe0f6..cfa0a318ff 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -123,17 +123,17 @@ nav: - user-guide/export/1_intro.md - user-guide/export/2_dataframes.md - user-guide/export/3_networkx.md - - GraphQL and User Interface: + - Making GraphQL queries: - user-guide/graphql/0_dummy_index.md - user-guide/graphql/1_intro.md - user-guide/graphql/2_run-server.md - user-guide/graphql/3_writing-queries.md - - user-guide/graphql/4_running-ui.md - Temporal graph representations: - user-guide/persistent-graph/0_dummy_index.md - user-guide/persistent-graph/1_intro.md - user-guide/persistent-graph/2_ambiguity.md - user-guide/persistent-graph/3_views.md + - user-guide/troubleshooting.md - Python API: reference/python/ - GraphQL API: reference/graphql/graphql_API.md - Rust API: https://docs.rs/raphtory/latest/raphtory/