privateGPT w/ AutoGPT

[Jawn78]

@imartinez has anyone been able to get autogpt to work with privateGPTs API? This would be awesome.

[Jawn78]

So I made some progress, but am now stuck.

If on AutoGPT you comment out the API key (and any ChatGPT settings) and set ```
OPENAI_API_BASE_URL=http://LocalIP:8001/v1
BACKEND_TYPE=PRIVATEGPT
The backend_type isn't anything official, they have some backends, but not GPT. However having this in the .env file seems to tell autogpt to use the OPENAI_API_BASE_URL

You will then get an error as AutoGPT attempts to check the model you are using. I suppressed this by commenting out the function  In Configurator.py  /autogpts/autogpt/app/configurator.py create_config

Set the default LLM models

if gpt3only:
    print_attribute("GPT3.5 Only Mode", "ENABLED")
    # --gpt3only should always use gpt-3.5-turbo, despite user's FAST_LLM config
    config.fast_llm = GPT_3_MODEL
    config.smart_llm = GPT_3_MODEL
elif (
    gpt4only
    and check_model(GPT_4_MODEL, model_type="smart_llm", config=config)
    == GPT_4_MODEL
):
    print_attribute("GPT4 Only Mode", "ENABLED")
    # --gpt4only should always use gpt-4, despite user's SMART_LLM config
    config.fast_llm = GPT_4_MODEL
    config.smart_llm = GPT_4_MODEL
# else:
#     config.fast_llm = check_model(config.fast_llm, "fast_llm", config=config)
#     config.smart_llm = check_model(config.smart_llm, "smart_llm", config=config)
else:
    print ("hello")

However, now privateGPT seems to have an issue with how autogpt is trying to communicate. It's a bit above my head and I don't have the time now to check it. Any advice would be great.

This is the error from privateGPT

11:43:19.881 [ERROR ] uvicorn.error - Exception in ASGI application
Traceback (most recent call last):
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 426, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in call
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\fastapi\applications.py", line 292, in call
await super().call(scope, receive, send)
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\starlette\applications.py", line 122, in call
await self.middleware_stack(scope, receive, send)
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\starlette\middleware\errors.py", line 184, in call
raise exc
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\starlette\middleware\errors.py", line 162, in call
await self.app(scope, receive, _send)
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\starlette\middleware\exceptions.py", line 79, in call
raise exc
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\starlette\middleware\exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in call
raise e
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in call
await self.app(scope, receive, send)
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\starlette\routing.py", line 718, in call
await route.handle(scope, receive, send)
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\starlette\routing.py", line 276, in handle
await self.app(scope, receive, send)
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\starlette\routing.py", line 66, in app
response = await func(request)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\fastapi\routing.py", line 273, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\fastapi\routing.py", line 192, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\starlette\concurrency.py", line 41, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\Documents\PrivateGPT2\privateGPT\private_gpt\server\chat\chat_router.py", line 81, in chat_completion
response = service.chat(all_messages, body.use_context, body.context_filter)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\Documents\PrivateGPT2\privateGPT\private_gpt\server\chat\chat_service.py", line 113, in chat
chat_response = self.llm_service.llm.chat(messages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\llama_index\llms\base.py", line 186, in wrapped_llm_chat
f_return_val = f(_self, messages, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\llama_index\llms\llama_cpp.py", line 203, in chat
prompt = self.messages_to_prompt(messages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Jawn78\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-9uCoDrym-py3.11\Lib\site-packages\llama_index\llms\llama_utils.py", line 32, in messages_to_prompt
assert user_message.role == MessageRole.USER
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

and this is the error from autoGPT
Traceback (most recent call last):
  File "C:\Users\jonat\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\openai\api_requestor.py", line 404, in handle_error_response
    error_data = resp["error"]
                 ~~~~^^^^^^^^^
TypeError: string indices must be integers, not 'str'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\jonat\OneDrive\Documents\AutoGPT\Auto-GPT\autogpts\autogpt\autogpt\__main__.py", line 5, in <module>
    autogpt.app.cli.main()
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\click\core.py", line 1666, in invoke
    rv = super().invoke(ctx)
         ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\autogpts\autogpt\autogpt\app\cli.py", line 121, in main
    run_auto_gpt(
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\autogpts\autogpt\autogpt\core\runner\client_lib\utils.py", line 59, in wrapper
    return asyncio.run(f(*args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\autogpts\autogpt\autogpt\app\main.py", line 199, in run_auto_gpt
    await run_interaction_loop(agent)
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\autogpts\autogpt\autogpt\app\main.py", line 309, in run_interaction_loop
    command_name, command_args, assistant_reply_dict = await agent.think()
                                                       ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\autogpts\autogpt\autogpt\agents\features\watchdog.py", line 36, in think
    command_name, command_args, thoughts = await super(WatchdogMixin, self).think(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\autogpts\autogpt\autogpt\agents\base.py", line 213, in think
    raw_response = await self.llm_provider.create_chat_completion(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\autogpts\autogpt\autogpt\core\resource\model_providers\openai.py", line 271, in create_chat_completion
    response = await self._create_chat_completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\autogpts\autogpt\autogpt\core\resource\model_providers\openai.py", line 448, in _wrapped
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\autogpts\autogpt\autogpt\core\resource\model_providers\openai.py", line 397, in _create_chat_completion
    return await openai.ChatCompletion.acreate(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\openai\api_resources\chat_completion.py", line 45, in acreate
    return await super().acreate(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 217, in acreate
    response, _, api_key = await requestor.arequest(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\openai\api_requestor.py", line 383, in arequest
    resp, got_stream = await self._interpret_async_response(result, stream)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\openai\api_requestor.py", line 729, in _interpret_async_response
    self._interpret_response_line(
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\openai\api_requestor.py", line 766, in _interpret_response_line
    raise self.handle_error_response(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jawn78\OneDrive\Documents\AutoGPT\Auto-GPT\venvAutoGPT\Lib\site-packages\openai\api_requestor.py", line 406, in handle_error_response
    raise error.APIError(
openai.error.APIError: Invalid response object from API: 'Internal Server Error' (HTTP response code was 500)

[rm-rf-etc]

HTTP response code was 500

Probably means the server was not reachable. Your URL looks weird to me, where did you get "http://localip:8001" from? Should be "localhost", not "localip"

OPENAI_API_BASE_URL=http://localhost:8001/v1

[Jawn78]

@rm-rf-etc, that was just me changing it from the IP address. It is pointing towards localhost