## I. WebLLM

The application supports WebLLM by default, a web-based interface for the LLM model.
WebLLM running locally in the browser provides a user-friendly interface for interacting with the LLM model. You can input text prompts, select model configurations, and generate text output in real-time.

**It totally **free** and **open-source**. You can access the WebLLM interface by visiting the following link: [WebLLM](https://github.com/mlc-ai/web-llm)**

There are several local LLM models available for use in the application. These models can be run locally on your machine, providing a fast and efficient way to generate text output without relying on external servers.

### 1. Qwen LLM

[![Qwen 2.5](./docs/qwen-2.5.png)](./docs/qwen-2.5.png)

Qwen, which refers to the large language model family built by Alibaba Cloud.

* Dense, easy-to-use, decoder-only language models, available in 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B sizes, and base and instruct variants.
* Pretrained on our latest large-scale dataset, encompassing up to 18T tokens.
* Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON.
* More resilient to the diversity of system prompts, enhancing role-play implementation and condition-setting for chatbots.
* Context length support up to 128K tokens and can generate up to 8K tokens.
* Multilingual support for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.

Latest release features the LLMs Qwen2.5, along with specialized models for coding, Qwen2.5-Coder, and mathematics, Qwen2.5-Math. All open-weight models are dense, decoder-only language models, available in various sizes, including:

* Qwen2.5: 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B
* Qwen2.5-Coder: 1.5B, 7B, and 32B on the way
* Qwen2.5-Math: 1.5B, 7B, and 72B.

### 2. Phi LLM

[![Phi 2.5](./docs/phi-3.jpg)](./docs/phi-3.jpg)

A family of powerful, small language models (SLMs) with groundbreaking performance at low cost and low latency

**Primary Use Cases**
The model is intended for commercial and research use in multiple languages. The model provides uses for general purpose AI systems and applications which require:

Memory/compute constrained environments
Latency bound scenarios
Strong reasoning (especially code, math and logic)
Our model is designed to accelerate research on language and multimodal models, for use as a building block for generative AI powered features.

**Use Case Considerations**
Our models are not specifically designed or evaluated for all downstream purposes. Developers should consider common limitations of language models as they select use cases, and evaluate and mitigate for accuracy, safety, and fariness before using within a specific downstream use case, particularly for high risk scenarios. Developers should be aware of and adhere to applicable laws or regulations (including privacy, trade compliance laws, etc.) that are relevant to their use case.

### 3. LLama 3.2

[![LLama 3.2](./docs/llama-3.2.jpg)](./docs/llama-3.2.jpg)

The lightweight 1B and 3B models are highly capable with multilingual text generation and tool calling abilities. These models empower developers to build personalized, on-device agentic applications with strong privacy where data never leaves the device. For example, such an application could help summarize the last 10 messages received, extract action items, and leverage tool calling to directly send calendar invites for follow-up meetings.

Running these models locally comes with two major advantages. First, prompts and responses can feel instantaneous, since processing is done locally. Second, running models locally maintains privacy by not sending data such as messages and calendar information to the cloud, making the overall application more private. Since processing is handled locally, the application can clearly control which queries stay on the device and which may need to be processed by a larger model in the cloud.

Our evaluation suggests that the Llama 3.2 vision models are competitive with leading foundation models, Claude 3 Haiku and GPT4o-mini on image recognition and a range of visual understanding tasks. The 3B model outperforms the Gemma 2 2.6B and Phi 3.5-mini models on tasks such as following instructions, summarization, prompt rewriting, and tool-use, while the 1B is competitive with Gemma.

We evaluated performance on over 150 benchmark datasets that span a wide range of languages. For the vision LLMs, we evaluated performance on benchmarks for image understanding and visual reasoning.


### 4. Hermes LLM

[![LLama 3.2](./docs/mistra-ai.svg)](./docs/mistra-ai.svg)

Hermes 2 Pro on Mistral 7B is the new flagship 7B Hermes!

Hermes 2 Pro is an upgraded, retrained version of Nous Hermes 2, consisting of an updated and cleaned version of the OpenHermes 2.5 Dataset, as well as a newly introduced Function Calling and JSON Mode dataset developed in-house.

This new version of Hermes maintains its excellent general task and conversation capabilities - but also excels at Function Calling, JSON Structured Outputs, and has improved on several other metrics as well, scoring a 90% on our function calling evaluation built in partnership with Fireworks.AI, and an 84% on our structured JSON Output evaluation.

Hermes Pro takes advantage of a special system prompt and multi-turn function calling structure with a new chatml role in order to make function calling reliable and easy to parse. Learn more about prompting below.


### 5. Gemma LLM

[![Gemma 2.5](./docs/gemma-logo-small.png)](./docs/gemma-logo-small.png)

Gemma is a family of 4 new LLM models by Google based on Gemini. It comes in two sizes: 2B and 7B parameters, each with base (pretrained) and instruction-tuned versions. All the variants can be run on various types of consumer hardware, even without quantization, and have a context length of 8K tokens:

* gemma-7b: Base 7B model.
* gemma-7b-it: Instruction fine-tuned version of the base 7B model.
* gemma-2b: Base 2B model.
* gemma-2b-it: Instruction fine-tuned version of the base 2B model.

### 6. Stable LM

[![StableLM](./docs/stable-lm.webp)](./docs/stable-lm.webp)

Experience cutting edge open access language models.

Boost your coding prowess with the efficiency of Stable Code, and venture beyond the horizon with our expansive selection of multilingual models, including the cutting-edge Stable LM Zephyr.

**More models, enjoy your exploration!**

## II. Wllama

Wllama enables running GUFF models directly in your browser using Hugging Face integration.

**Key Points:**
- Supports running GUFF models from Hugging Face.
- Designed for efficient local inference in the browser.
- Ideal for lightweight and experimental deployments.

**Notice:**
Only works with models smaller than 2GB. Models larger than 2GB require splitting and are not supported directly.
WebGPU is not yet supported; only CPU inference is available.

## III. OpenAI

### How to Get Your OpenAI API Key

To use OpenAI models, you need an API key from OpenAI. Follow these steps:

1. Go to [OpenAI Platform](https://platform.openai.com/).
2. Sign in or create an account.
3. Navigate to 'API Keys' in your account settings.
4. Click 'Create new secret key' and copy the generated key.

**Security Notice:**
Your OpenAI API key is stored locally and encrypted with your personal encryption key. This ensures your API key remains private and secure on your device.

Never share your API key publicly or commit it to version control.

## IV. Google Generative AI

Google Generative AI provides access to advanced models like Gemini Pro and Gemini Ultra for text, code, and multimodal tasks.

**Key Features:**
- Multimodal capabilities (text, image, code)
- High performance for reasoning and creative tasks
- Available via Google AI Studio and API

**How to Get Started:**
1. Visit [Google AI Studio](https://ai.google.com/).
2. Sign in with your Google account.
3. Create a new API key in the developer console.
4. Use the key in your application to access Gemini models.

**Security Notice:**
Your Google Generative AI API key is stored locally and encrypted with your personal encryption key for privacy and security.

## V. Groq AI

Groq provides ultra-fast access to leading language models such as Llama-3, Mixtral, and Gemma via its API, optimized for low latency and high throughput.

**Key Features:**
- Extremely low latency responses
- Supports top open-source models (Llama-3, Mixtral, Gemma, etc.)
- Scalable and reliable API for production use

**How to Get Started:**
1. Visit [Groq Cloud Console](https://console.groq.com/).
2. Sign in or create a Groq account.
3. Go to 'API Keys' and generate a new key.
4. Use the key in your application to access Groq-hosted models.

**Security Notice:**
Your Groq API key is stored locally and encrypted with your personal encryption key for privacy and security.

## Additional Resources
- [Embedding](./#/document/embedding "<_self>")
- [Get started Example](./#/document/get-started "<_self>")

<br />
<br />
<br />
<br />
<br />

<br />
<br />
<br />
<br />
<br />
