File Content Extraction 25.2
Filter SDK .NET Programming Guide

# Character Encoding

To ensure that all filtered text is output in the same character encoding, File Content Extraction performs character encoding conversion. In most cases, if your license includes advanced character set detection, File Content Extraction can detect the character encoding used in a source file, and automatically outputs filtered text in the encoding you choose. OpenText recommends that you specify your preferred target encoding. In the rare cases where File Content Extraction cannot detect the character encoding used in a source file, you can also specify the source encoding.

Many encodings have significant overlap in terms of the bytes used to encode characters, with the same bytes used to express different characters in different encodings. Advanced character set detection performs frequency analysis on the characters of the text to determine the most likely language and encoding. Detection therefore works best on large portions of natural language, and might not be able to determine the correct source encoding where the text is very short, contains a random sequence of characters, has some blocks of characters repeated many times, or contains characters from multiple languages.

## Specify a Target Character Encoding

The .NET API does not expose an option for the target character encoding. The output encoding is always UTF-8.

## Performance Considerations

When a file format does not specify a character encoding, File Content Extraction attempts to detect the encoding automatically. Some character encodings, including UTF-8 and UTF-16, can be detected by core File Content Extraction functionality but others can be detected only if your license includes advanced character set detection. Advanced character set detection is enabled by default (if it is included in your license), but can increase the time required to filter some documents.

You can disable advanced character set detection on a file-by-file basis. Before doing this, be aware that File Content Extraction cannot output filtered text in your chosen encoding unless it detects the encoding of the source file, or you specify the source encoding yourself though the API.

To disable advanced character set detection

- In the .NET API, call the method `CharacterSetDetection` on your session configuration:

      session.Config().CharacterSetDetection(false);

## Specify a Source Character Encoding

In most cases, File Content Extraction can automatically detect the character encoding of an input file and specifying a source encoding is not necessary. You might need to specify the source character encoding if you have disabled advanced character set detection.

To specify the source character encoding

- In the .NET API, call the method `SourceCharset` on your session configuration.

      session.Config().SourceCharset(CharSet.KVCS_8859_1);

Copyright 1997-2025 Open Text
Version 25.2 | Last updated April 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: File Content Extraction 25.2

Topic Title: Character Encoding