---
title: "Reference: Unicode Normalizer | Processors"
description: "Documentation for the UnicodeNormalizer in Mastra, which normalizes Unicode text to ensure consistent formatting and remove potentially problematic characters."
packages:
  - "@mastra/core"
---

# UnicodeNormalizer

The `UnicodeNormalizer` is an **input processor** that normalizes Unicode text to ensure consistent formatting and remove potentially problematic characters before messages are sent to the language model. This processor helps maintain text quality by handling various Unicode representations, removing control characters, and standardizing whitespace formatting.

## Usage example

```typescript
import { UnicodeNormalizer } from "@mastra/core/processors";

const processor = new UnicodeNormalizer({
  stripControlChars: true,
  collapseWhitespace: true
});
```

## Constructor parameters

<PropertiesTable
  content={[
    {
      name: "options",
      type: "Options",
      description: "Configuration options for Unicode text normalization",
      isOptional: true,
    },
  ]}
/>

### Options

<PropertiesTable
  content={[
    {
      name: "stripControlChars",
      type: "boolean",
      description: "Whether to strip control characters. When enabled, removes control characters except \t, \n, \r",
      isOptional: true,
      default: "false",
    },
    {
      name: "preserveEmojis",
      type: "boolean",
      description: "Whether to preserve emojis. When disabled, emojis may be removed if they contain control characters",
      isOptional: true,
      default: "true",
    },
    {
      name: "collapseWhitespace",
      type: "boolean",
      description: "Whether to collapse consecutive whitespace. When enabled, multiple spaces/tabs/newlines are collapsed to single instances",
      isOptional: true,
      default: "true",
    },
    {
      name: "trim",
      type: "boolean",
      description: "Whether to trim leading and trailing whitespace",
      isOptional: true,
      default: "true",
    },
  ]}
/>

## Returns

<PropertiesTable
  content={[
    {
      name: "id",
      type: "string",
      description: "Processor identifier set to 'unicode-normalizer'",
      isOptional: false,
    },
    {
      name: "name",
      type: "string",
      description: "Optional processor display name",
      isOptional: true,
    },
    {
      name: "processInput",
      type: "(args: { messages: MastraDBMessage[]; abort: (reason?: string) => never }) => MastraDBMessage[]",
      description: "Processes input messages to normalize Unicode text",
      isOptional: false,
    },
  ]}
/>


## Extended usage example

```typescript title="src/mastra/agents/normalized-agent.ts"
import { Agent } from "@mastra/core/agent";
import { UnicodeNormalizer } from "@mastra/core/processors";

export const agent = new Agent({
  id: "normalized-agent",
  name: "normalized-agent",
  instructions: "You are a helpful assistant",
  model: "openai/gpt-5.1",
  inputProcessors: [
    new UnicodeNormalizer({
      stripControlChars: true,
      preserveEmojis: true,
      collapseWhitespace: true,
      trim: true
    })
  ]
});
```


## Related

- [Guardrails](/docs/v1/agents/guardrails)
