How to Generate h2oGPT API responses with Python

So you installed your own h2oGPT on Windows, and you want to call it via its API. How to generate h2oGPT API responses programmatically with local API calls? There is a basic example provided on Github, but here is a Python3 script that will do it for you! There is a more powerful client called h2ogpt_client but as part of the h2oai project, you cannot install it on Windows yet.

Requisites

• miniconda3: a multi-Python environment manager
• h2oGPT for Windows installed, with all requirements needed, including AutoGPTQ for GPU
• create an environment for your h2oGPT: conda create -n h2ogpt -y
• activate it: conda activate h2ogpt
• h2oGPT currently works with Python3.10

The Python Script to call h2oGPT API

Create a Python script api.py inside h2ogpt folder:

import sys, getopt
from gradio_client import Client
import ast
from pprint import pprint

HOST_URL = "http://localhost:7860"
question = 'Who are you?'
inputfile = ''
outputfile = ''

def main(argv):
    global HOST_URL
    global question
    global inputfile
    global outputfile
    opts, args = getopt.getopt(argv,"hq:i:o:",["ifile=","ofile="])
    for opt, arg in opts:
      if opt == '-h':
        print ('test.py -i <inputfile> -o <outputfile>')
        exit()
        sys.exit()
      elif opt in ("-i", "--ifile"):
        inputfile = arg
      elif opt in ("-o", "--ofile"):
        outputfile = arg
      elif opt in ("-q", "--qquestion"):
        question = arg


if __name__ == "__main__":
   main(sys.argv[1:])

# https://github.com/h2oai/h2ogpt/blob/main/docs/README_CLIENT.md
print('question:',question)
client = Client(HOST_URL)
# string of dict for input
kwargs = dict(instruction_nochat=question)
res = client.predict(str(dict(kwargs)), api_name='/submit_nochat_api')
        # REM assert langchain_mode in langchain_modes, "Invalid langchain_mode %s" % langchain_mode
    # REM AssertionError: Invalid langchain_mode Disabled

# string of dict for output
response = ast.literal_eval(res)['response']
print('response:',response)

How to use h2oGPT API script to speak with your Bot

command: python api.py -q "question"

Example:

python api.py -q "About Arizona state legislature A.R.S. §33-1321 that covers security deposits: how long does the landlord have to send the security deposit back after vacancy?

Output:

question: About Arizona state legislature A.R.S. §33-1321 that covers security deposits: how long does the landlord have to send the security deposit back after vacancy?
Loaded as API: http://localhost:7860/ ✔
response: The Arizona state legislature A.R.S. §33-1321 states that a landlord must return a tenant's security deposit within 21 days after the tenant vacates the rental property. If the landlord fails to return the deposit within this time frame, the tenant may file a claim with the Arizona Department of Housing to recover the deposit.

❌

◀️ ▶️