Potential Improvements and Refinements

To the lab materials

If you have any feedback regarding this Lab, please use Issues and Pull Requests.

But this section of the lab is not meant for Improvements and Refinements to the lab. Instead it is about…​

To the presented prototype

What we have shown in this lab is a very rough prototype, put together very quickly, in order to demonstrate:

  • which improvements could be done

  • how long it would take to do them

In such a situation, it is common to go fast and make short-term decisions since there is no guarantee that this will become a real project.

Therefore, now, we need to review what was done, and make suggestions along these axes.

  • What other tools and techniques could be used in order to make the experience of our claims adjusters even better, more streamlined?

  • Is our current application robust enough? Describe what scenarios might make it harder to use?

  • How efficiently are we going to be able to make updates to its components if we need to? Is there a chance that making a change to the LLM model, the YOLOV8 model, the database, the application, will introduce breaking changes?

  • If that is the case, what would we want to do to avoid those risks?

  • Which parts seem to still be manual and time-consuming, and how could we automate them better, to avoid human errors if we can.

Think about these questions first

If you want to read what we thought could be improved, read below! (responses are not exhaustive)

Details

  • We could have something that analyzes the images and checks for discrepancies with the customer data, such as:

    • not the same make or color car as what is on file

    • mismatch in license plate, if visible in the picture

  • We’ve only scratched the surface with gitops and pipelines here

    • There was no performance testing done. If too many users connect at the same time, it might overwhelm either the app, the database, the LLM, etc…​

  • Currently, most simple changes would probably end up breaking the application. And the person who, for example decides to change Mistral7B for Flan-T5-Small would not necessarily realize that.

    • It would be critical to have multiple instances (Dev/Test/UAT/Prod) of the application

    • It would also be required to have integration pipelines run in these environments to confirm that changes made do not break the overall application.

  • We could ask the LLM to start writing a response to the customer.

    • It could be just to ask for missing details.

    • or it could be to let them know whether the claim is accepted or denied

  • However, to do this, the LLM would have to be aware of the policies that the insurance company uses to make those determinations.

    • This could be an interesting use-case for the RAG approach.