So you installed your own h2oGPT on Windows, and you want to call it via its API. How to generate h2oGPT API responses programmatically with local API calls? There is a basic example provided on Github, but here is a Python3 script that will do it for you! There is a more powerful client called h2ogpt_client but as part of the h2oai project, you cannot install it on Windows yet.

Requisites

• miniconda3: a multi-Python environment manager
• h2oGPT for Windows installed, with all requirements needed, including AutoGPTQ for GPU
• create an environment for your h2oGPT: conda create -n h2ogpt -y
• activate it: conda activate h2ogpt
• h2oGPT currently works with Python3.10

The Python Script to call h2oGPT API

Create a Python script api.py inside h2ogpt folder:

import sys, getopt
from gradio_client import Client
import ast
from pprint import pprint

HOST_URL = "http://localhost:7860"
question = 'Who are you?'
inputfile = ''
outputfile = ''

def main(argv):
    global HOST_URL
    global question
    global inputfile
    global outputfile
    opts, args = getopt.getopt(argv,"hq:i:o:",["ifile=","ofile="])
    for opt, arg in opts:
      if opt == '-h':
        print ('test.py -i <inputfile> -o <outputfile>')
        exit()
        sys.exit()
      elif opt in ("-i", "--ifile"):
        inputfile = arg
      elif opt in ("-o", "--ofile"):
        outputfile = arg
      elif opt in ("-q", "--qquestion"):
        question = arg


if __name__ == "__main__":
   main(sys.argv[1:])

# https://github.com/h2oai/h2ogpt/blob/main/docs/README_CLIENT.md
print('question:',question)
client = Client(HOST_URL)
# string of dict for input
kwargs = dict(instruction_nochat=question)
res = client.predict(str(dict(kwargs)), api_name='/submit_nochat_api')
        # REM assert langchain_mode in langchain_modes, "Invalid langchain_mode %s" % langchain_mode
    # REM AssertionError: Invalid langchain_mode Disabled

# string of dict for output
response = ast.literal_eval(res)['response']
print('response:',response)

How to use h2oGPT API script to speak with your Bot

command: python api.py -q "question"

Example:

python api.py -q "About Arizona state legislature A.R.S. §33-1321 that covers security deposits: how long does the landlord have to send the security deposit back after vacancy?

Output:

question: About Arizona state legislature A.R.S. §33-1321 that covers security deposits: how long does the landlord have to send the security deposit back after vacancy?
Loaded as API: http://localhost:7860/ ✔
response: The Arizona state legislature A.R.S. §33-1321 states that a landlord must return a tenant's security deposit within 21 days after the tenant vacates the rental property. If the landlord fails to return the deposit within this time frame, the tenant may file a claim with the Arizona Department of Housing to recover the deposit.