Jarvis (HuggingGPT)

A unique collaborative system has been developed by Microsoft, which employs multiple AI models to accomplish a given task, with ChatGPT functioning as the task controller. This project, known as JARVIS on GitHub (visit), is now available on Huggingface as HuggingGPT for people to experiment with. In our tests, the system performed exceptionally well with various data types, including texts, images, audio, and even videos.

The approach taken by JARVIS is similar to that of OpenAI's demonstration of GPT-4's multimodal capabilities with texts and images, but JARVIS takes it a step further by integrating several open-source LLMs for images, videos, audio, and more. What's even more impressive is that it can also connect to the internet and access files. For example, it can accept a URL from a website and answer questions about it.