Update Documentation (#331)

* Update documentation

* Add API reference for new readers

* Add example to BigQueryReader

* Add custom operator tutorial

* Add example to DuckDBReader

Author: Dominastorm

docs/api-reference/operators/IO/BigQueryReader.md

@@ -0,0 +1 @@
+::: uptrain.operators.BigQueryReader

docs/api-reference/operators/IO/DuckDBReader.md

@@ -0,0 +1 @@
+::: uptrain.operators.DuckDBReader

docs/api-reference/operators/IO/ExcelReader.md

@@ -0,0 +1 @@
+::: uptrain.operators.ExcelReader

…rators/code/SQL/ExecuteAndComparseSQL.md …erators/code/SQL/ExecuteAndCompareSQL.mddocs/api-reference/operators/code/SQL/ExecuteAndComparseSQL.md renamed to docs/api-reference/operators/code/SQL/ExecuteAndCompareSQL.md docs/api-reference/operators/code/SQL/ExecuteAndComparseSQL.md renamed to docs/api-reference/operators/code/SQL/ExecuteAndCompareSQL.md

@@ -16,3 +16,5 @@ Each Operator implements two methods, which are to be run in order:
 Though operators can be used on their own, they are typically used in [Checks](/key-components/check), that make it easy to add [Charts](/key-components/chart) chain multiple operators together.  Using a `Check`, you won't have to manually perform the `setup()` and `run()` methods.  
 
 To learn more about Operators, see the [Operator documentation](https://uptrain-ai.github.io/uptrain/operators/Accuracy/).
+
+You can also create your own custom operators.  To learn more, see the [Custom Operator tutorial](/tutorials/custom-operator).

docs/key-components/operator.mdx

@@ -78,7 +78,8 @@
       "pages": [
         "tutorials/prompt-experiments",
         "tutorials/validation",
-        "tutorials/openai-evals"
+        "tutorials/openai-evals",
+        "tutorials/custom-operator"
       ]
     },
     {

docs/mint.json

@@ -0,0 +1,92 @@
+---
+title: Custom Operator
+description: Learn how to create your own custom operator
+---
+
+## What are Custom Operators?
+UpTrain offers a wide range of built-in operators to help you get started with your training pipeline. However, you may want to create your own custom operator to perform a specific task. This tutorial will walk you through the process of creating a custom operator.
+
+There are a few things that need to be kept in mind while creating an operator:
+1. Operators have two methods: `setup` and `run`
+    1. `setup` is called once when the operator is initialized. This is where you can pass in any settings that you need to use in the `run` method.
+    2. `run` is called for each batch of data that is passed to the operator. This is where you will perform the actual operation on the data. It returns a dictionary with the key `output` and the value depends on the type of operator. Any extra information can be put in the `extra` key of the dictionary.
+2. There are two types of operators: `TransformOp` and `ColumnOp`. 
+    1. `TransformOp` represents an operator that transforms the data into another form. 
+        - This is used for operations like filtering, cleaning, etc. 
+        - The value of the `output` key of the dictionary returned by the `run` method should be a polars.DataFrame or None.
+    2. `ColumnOp` represents an operator that adds a new column to the data. 
+        - This is used for operations like adding a new column, renaming a column, etc. 
+        - The value of the `output` key of the dictionary returned by the `run` method should set as the computed table.
+3. The operator should be registered using the `register_custom_op` decorator. 
+
+## Examples
+
+#### Example 1: Cleanup Operator
+
+An Operator that goes through a list of messages and extracts the question, document title, document link, document text, and response from the messages.
+```python
+from uptrain.operators import TransformOp, register_custom_op
+
+@register_custom_op
+class Cleanup(TransformOp):
+    def setup(self, settings):
+        return self
+
+    def run(self, dataset):
+        import json
+        import polars as pl
+
+        table_cols = [ 
+            "question",
+            "document_title",
+            "document_link",
+            "document_text",
+            "response",
+        ]
+        out = []
+        for point in dataset.to_dicts():
+            messages = json.loads(point["messages"])
+            question = messages[0]["content"].split("The input is: '")[1].split("?")[0]
+            name = (
+                messages[0]["content"]
+                .split("technical documentation titled ")[1]
+                .split(", found at")[0]
+            )
+            link = messages[0]["content"].split("found at ")[1].split(". \n")[0]
+            text = (
+                messages[0]["content"]
+                .split("--- START: Document ---")[1]
+                .split(name + "\n")[1]
+                .split("\n\n--- END: Document")[0]
+            )
+            response = messages[1]["content"][1:-1]
+
+            new_row = dict(zip(table_cols, [question, name, link, text, response]))
+            out.append(new_row)
+
+        return {"output": pl.from_dicts(out)}
+```
+
+#### Example 2: AddContext Operator 
+
+An Operator that adds the model and pipeline name to the data.
+```python
+from uptrain.operators import TransformOp, register_custom_op
+
+@register_custom_op
+class AddContext(TransformOp):
+    def setup(self, settings):
+        return self
+
+    def run(self, dataset):
+        import polars as pl
+
+        return {
+            "output": dataset.with_columns(
+                [
+                    pl.lit("gpt-4").alias("model"),
+                    pl.lit("context_retrieval").alias("pipeline"),
+                ]
+            )
+        }
+```

docs/tutorials/custom-operator.mdx

@@ -54,7 +54,8 @@ __all__ = [
     "JsonWriter",
     "DeltaReader",
     "DeltaWriter",
-    "BigqueryReader",
+    "BigQueryReader",
+    "DuckDBReader",
     # code
     "code",
     "ParseCreateStatements",
@@ -104,7 +105,8 @@ from .language.generation import PromptGenerator, TextCompletion, OutputParser
 from . import io
 from .io.base import CsvReader, JsonReader, DeltaReader, JsonWriter, DeltaWriter
 from .io.excel import ExcelReader
-from .io.bq import BigqueryReader
+from .io.bq import BigQueryReader
+from .io.duck import DuckDBReader
 
 from . import code
 from .code.sql import (

uptrain/operators/__init__.pyi

@@ -15,7 +15,7 @@
 
 
 @register_op
-class BigqueryReader(TransformOp):
+class BigQueryReader(TransformOp):
     """Read data from a bigquery table.
 
     NOTE: To use this operator, you must include the GCP service account credentials in
@@ -24,6 +24,18 @@ class BigqueryReader(TransformOp):
     Attributes:
         query (str): Query to run against the BigQuery table.
         col_timestamp (str): Column name to use as the timestamp column. Only used in the context of monitoring.
+
+    Example:
+        ```python
+        from uptrain.operators.io import BigQueryReader
+
+        query = "SELECT * FROM `bigquery-public-data.samples.shakespeare` LIMIT 10"
+        reader = BigQueryReader(
+            query=query,
+            col_timestamp="timestamp"
+        )
+        output = reader.setup().run()["output"]
+        ```
     """
 
     query: str

uptrain/operators/io/bq.py

@@ -15,13 +15,25 @@
 
 
 @register_op
-class DuckdbReader(TransformOp):
+class DuckDBReader(TransformOp):
     """Read data from a Duckdb table.
 
     Attributes:
         fpath (str): Path to the Duckdb file.
         query (str): Query to run against the duckdb database.
         col_timestamp (str): Column name to use as the timestamp column. Only used in the context of monitoring.
+
+    Example:
+        ```python
+        from uptrain.operators.io import DuckDBReader
+
+        reader = DuckDBReader(
+            fpath="data/duckdb.db",
+            query="SELECT * FROM my_table",
+            col_timestamp="timestamp",
+        )
+        output = reader.setup().run()["output"]
+        ```
     """
 
     fpath: str