How to generate h2oGPT API responses programmatically with Python3 and local API calls

Spread the love
(Last Updated On: August 23, 2023)

So you installed your own h2oGPT on Windows, and you want to call it via its internal API. How to generate h2oGPT API responses programmatically with local API calls? There is a basic example provided on Github, but here is a Python3 script that will do it for you! There is a more powerful client called h2ogpt_client but as part of the h2oai project, you cannot install it on Windows yet.

 

Requisites

  • miniconda3: a multi-Python environment manager
  • h2oGPT for Windows installed, with all requirements needed, including AutoGPTQ for GPU
  • create an environment for your h2oGPT: conda create -n h2ogpt -y
  • activate it: conda activate h2ogpt
  • h2oGPT currently works with Python3.10

 

The script to call h2oGPT API /submit_nochat_api

Create a Python script api.py inside h2ogpt folder:

import sys, getopt
from gradio_client import Client
import ast
from pprint import pprint

HOST_URL = "http://localhost:7860"
question = 'Who are you?'
inputfile = ''
outputfile = ''

def main(argv):
    global HOST_URL
    global question
    global inputfile
    global outputfile
    opts, args = getopt.getopt(argv,"hq:i:o:",["ifile=","ofile="])
    for opt, arg in opts:
      if opt == '-h':
        print ('test.py -i <inputfile> -o <outputfile>')
        exit()
        sys.exit()
      elif opt in ("-i", "--ifile"):
        inputfile = arg
      elif opt in ("-o", "--ofile"):
        outputfile = arg
      elif opt in ("-q", "--qquestion"):
        question = arg


if __name__ == "__main__":
   main(sys.argv[1:])

# https://github.com/h2oai/h2ogpt/blob/main/docs/README_CLIENT.md
print('question:',question)
client = Client(HOST_URL)
# string of dict for input
kwargs = dict(instruction_nochat=question)
res = client.predict(str(dict(kwargs)), api_name='/submit_nochat_api')
        # REM assert langchain_mode in langchain_modes, "Invalid langchain_mode %s" % langchain_mode
    # REM AssertionError: Invalid langchain_mode Disabled

# string of dict for output
response = ast.literal_eval(res)['response']
print('response:',response)

 

How to use the h2oGPT API script to speak with your h2oGPT

command: python api.py -q "question"

Example:

python api.py -q "About Arizona state legislature A.R.S. §33-1321 that covers security deposits: how long does the landlord have to send the security deposit back after vacancy?

Output:

question: About Arizona state legislature A.R.S. §33-1321 that covers security deposits: how long does the landlord have to send the security deposit back after vacancy?
Loaded as API: http://localhost:7860/ ✔
response: The Arizona state legislature A.R.S. §33-1321 states that a landlord must return a tenant's security deposit within 21 days after the tenant vacates the rental property. If the landlord fails to return the deposit within this time frame, the tenant may file a claim with the Arizona Department of Housing to recover the deposit.

 

 

Team Cook

Bachelor in IT, father, Photograph, Communist slayer, I also build useful tools for my community and destroy repetitive tasks for the past 20 years. "Who controls the information, controls the future."

0 0 votes
Article Rating
Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x