Backend for the flutter app Flutter App
Project for the Google Challange of the START Hack 2023
- the backend in written in python
- FastAPI Framework for the REST API
- OpenAI GPT-3
- Google Cloud Vision API
Install the python requirements with:
pip install -r requirements.txt
- you need an API access to OpenAI
- you need access to the google cloud
The frontend (Flutter App) sends us an image of the receipt. We send this image to the google cloud vision api and get back the text that is on the image. We send this text together with a predefined query to the OpenAI API (gpt-3.5-turbo model). We have designed the query so that we now get the relevant data of the receipt converted as JSON back (in controller.py is exact query). Also GPT matches the items of the receipt to our existing in the database (e.g. Milka -> Chocolate). After that we match the results with our CO2 database and send this data back to the frontend.
More informations about the use / way of the app: Frontend Flutter App GitHub