# class Encoding

An `Encoding` instance represents a character encoding usable in Ruby. It is defined as a constant under the `Encoding` namespace. It has a name and optionally, aliases:

    Encoding::ISO_8859_1.name
    #=> "ISO-8859-1"

    Encoding::ISO_8859_1.names
    #=> ["ISO-8859-1", "ISO8859-1"]

Ruby methods dealing with encodings return or accept `Encoding` instances as arguments (when a method accepts an `Encoding` instance as an argument, it can be passed an `Encoding` name or alias instead).

    "some string".encoding
    #=> #<Encoding:UTF-8>

    string = "some string".encode(Encoding::ISO_8859_1)
    #=> "some string"
    string.encoding
    #=> #<Encoding:ISO-8859-1>

    "some string".encode "ISO-8859-1"
    #=> "some string"

Encoding::ASCII_8BIT is a special encoding that is usually used for a byte string, not a character string. But as the name insists, its characters in the range of ASCII are considered as ASCII characters. This is useful when you use ASCII-8BIT characters with other ASCII compatible characters.

## Changing an encoding¶ ↑

The associated `Encoding` of a `String` can be changed in two different ways.

First, it is possible to set the `Encoding` of a string to a new `Encoding` without changing the internal byte representation of the string, with `String#force_encoding`. This is how you can tell Ruby the correct encoding of a string.

    string
    #=> "R\xC3\xA9sum\xC3\xA9"
    string.encoding
    #=> #<Encoding:ISO-8859-1>
    string.force_encoding(Encoding::UTF_8)
    #=> "R\u00E9sum\u00E9"

Second, it is possible to transcode a string, i.e. translate its internal byte representation to another encoding. Its associated encoding is also set to the other encoding. See `String#encode` for the various forms of transcoding, and the `Encoding::Converter` class for additional control over the transcoding process.

    string
    #=> "R\u00E9sum\u00E9"
    string.encoding
    #=> #<Encoding:UTF-8>
    string = string.encode!(Encoding::ISO_8859_1)
    #=> "R\xE9sum\xE9"
    string.encoding
    #=> #<Encoding::ISO-8859-1>

## Script encoding¶ ↑

All Ruby script code has an associated `Encoding` which any `String` literal created in the source code will be associated to.

The default script encoding is Encoding::UTF_8 after v2.0, but it can be changed by a magic comment on the first line of the source code file (or second line, if there is a shebang line on the first). The comment must contain the word `coding` or `encoding`, followed by a colon, space and the `Encoding` name or alias:

    # encoding: UTF-8

    "some string".encoding
    #=> #<Encoding:UTF-8>

The `__ENCODING__` keyword returns the script encoding of the file which the keyword is written:

    # encoding: ISO-8859-1

    __ENCODING__
    #=> #<Encoding:ISO-8859-1>

`ruby -K` will change the default locale encoding, but this is not recommended. Ruby source files should declare its script encoding by a magic comment even when they only depend on US-ASCII strings or regular expressions.

## Locale encoding¶ ↑

The default encoding of the environment. Usually derived from locale.

see `Encoding.locale_charmap`, `Encoding.find`(‘locale’)

## Filesystem encoding¶ ↑

The default encoding of strings from the filesystem of the environment. This is used for strings of file names or paths.

see `Encoding.find`(‘filesystem’)

## External encoding¶ ↑

Each `IO` object has an external encoding which indicates the encoding that Ruby will use to read its data. By default Ruby sets the external encoding of an `IO` object to the default external encoding. The default external encoding is set by locale encoding or the interpreter `-E` option. `Encoding.default_external` returns the current value of the external encoding.

    ENV["LANG"]
    #=> "UTF-8"
    Encoding.default_external
    #=> #<Encoding:UTF-8>

    $ ruby -E ISO-8859-1 -e "p Encoding.default_external"
    #<Encoding:ISO-8859-1>

    $ LANG=C ruby -e 'p Encoding.default_external'
    #<Encoding:US-ASCII>

The default external encoding may also be set through `Encoding.default_external=`, but you should not do this as strings created before and after the change will have inconsistent encodings. Instead use `ruby -E` to invoke ruby with the correct external encoding.

When you know that the actual encoding of the data of an `IO` object is not the default external encoding, you can reset its external encoding with `IO#set_encoding` or set it at `IO` object creation (see `IO.new` options).

## Internal encoding¶ ↑

To process the data of an `IO` object which has an encoding different from its external encoding, you can set its internal encoding. Ruby will use this internal encoding to transcode the data when it is read from the `IO` object.

Conversely, when data is written to the `IO` object it is transcoded from the internal encoding to the external encoding of the `IO` object.

The internal encoding of an `IO` object can be set with `IO#set_encoding` or at `IO` object creation (see `IO.new` options).

The internal encoding is optional and when not set, the Ruby default internal encoding is used. If not explicitly set this default internal encoding is `nil` meaning that by default, no transcoding occurs.

The default internal encoding can be set with the interpreter option `-E`. `Encoding.default_internal` returns the current internal encoding.

    $ ruby -e 'p Encoding.default_internal'
    nil

    $ ruby -E ISO-8859-1:UTF-8 -e "p [Encoding.default_external, \
      Encoding.default_internal]"
    [#<Encoding:ISO-8859-1>, #<Encoding:UTF-8>]

The default internal encoding may also be set through `Encoding.default_internal=`, but you should not do this as strings created before and after the change will have inconsistent encodings. Instead use `ruby -E` to invoke ruby with the correct internal encoding.

## `IO` encoding example¶ ↑

In the following example a UTF-8 encoded string “Ru00E9sumu00E9” is transcoded for output to ISO-8859-1 encoding, then read back in and transcoded to UTF-8:

    string = "R\u00E9sum\u00E9"

    open("transcoded.txt", "w:ISO-8859-1") do |io|
      io.write(string)
    end

    puts "raw text:"
    p File.binread("transcoded.txt")
    puts

    open("transcoded.txt", "r:ISO-8859-1:UTF-8") do |io|
      puts "transcoded text:"
      p io.read
    end

While writing the file, the internal encoding is not specified as it is only necessary for reading. While reading the file both the internal and external encoding must be specified to obtain the correct result.

    $ ruby t.rb
    raw text:
    "R\xE9sum\xE9"

    transcoded text:
    "R\u00E9sum\u00E9"

### Public Class Methods

aliases -> {"alias1" => "orig1", "alias2" → "orig2", ...} click to toggle source

Returns the hash of available encoding alias and original encoding name.

    Encoding.aliases
    #=> {"BINARY"=>"ASCII-8BIT", "ASCII"=>"US-ASCII", "ANSI_X3.4-1968"=>"US-ASCII",
          "SJIS"=>"Windows-31J", "eucJP"=>"EUC-JP", "CP932"=>"Windows-31J"}

    static VALUE
    rb_enc_aliases(VALUE klass)
    {
        VALUE aliases[2];
        aliases[0] = rb_hash_new();
        aliases[1] = rb_ary_new();

        GLOBAL_ENC_TABLE_EVAL(enc_table,
                              st_foreach(enc_table->names, rb_enc_aliases_enc_i, (st_data_t)aliases));

        return aliases[0];
    }

compatible?(obj1, obj2) → enc or nil click to toggle source

Checks the compatibility of two objects.

If the objects are both strings they are compatible when they are concatenatable. The encoding of the concatenated string will be returned if they are compatible, nil if they are not.

    Encoding.compatible?("\xa1".force_encoding("iso-8859-1"), "b")
    #=> #<Encoding:ISO-8859-1>

    Encoding.compatible?(
      "\xa1".force_encoding("iso-8859-1"),
      "\xa1\xa1".force_encoding("euc-jp"))
    #=> nil

If the objects are non-strings their encodings are compatible when they have an encoding and:

- Either encoding is US-ASCII compatible

- One of the encodings is a 7-bit encoding

    static VALUE
    enc_compatible_p(VALUE klass, VALUE str1, VALUE str2)
    {
        rb_encoding *enc;

        if (!enc_capable(str1)) return Qnil;
        if (!enc_capable(str2)) return Qnil;
        enc = rb_enc_compatible(str1, str2);
        if (!enc) return Qnil;
        return rb_enc_from_encoding(enc);
    }

default_external → enc click to toggle source

Returns default external encoding.

The default external encoding is used by default for strings created from the following locations:

- `CSV`

- `File` data read from disk

- SDBM

- `StringIO`

- `Zlib::GzipReader`

- `Zlib::GzipWriter`

- `String#inspect`

- `Regexp#inspect`

While strings created from these locations will have this encoding, the encoding may not be valid. Be sure to check `String#valid_encoding?`.

`File` data written to disk will be transcoded to the default external encoding when written, if `default_internal` is not nil.

The default external encoding is initialized by the -E option. If -E isn’t set, it is initialized to UTF-8 on Windows and the locale on other operating systems.

    static VALUE
    get_default_external(VALUE klass)
    {
        return rb_enc_default_external();
    }

default_external = enc click to toggle source

Sets default external encoding. You should not set `Encoding::default_external` in ruby code as strings created before changing the value may have a different encoding from strings created after the value was changed., instead you should use `ruby -E` to invoke ruby with the correct default_external.

See `Encoding::default_external` for information on how the default external encoding is used.

    static VALUE
    set_default_external(VALUE klass, VALUE encoding)
    {
        rb_warning("setting Encoding.default_external");
        rb_enc_set_default_external(encoding);
        return encoding;
    }

default_internal → enc click to toggle source

Returns default internal encoding. Strings will be transcoded to the default internal encoding in the following places if the default internal encoding is not nil:

- `CSV`

- `Etc.sysconfdir` and `Etc.systmpdir`

- `File` data read from disk

- `File` names from `Dir`

- `Integer#chr`

- `String#inspect` and `Regexp#inspect`

- Strings returned from `Readline`

- Strings returned from SDBM

- `Time#zone`

- Values from `ENV`

- Values in ARGV including $PROGRAM_NAME

Additionally `String#encode` and `String#encode!` use the default internal encoding if no encoding is given.

The script encoding (__ENCODING__), not `default_internal`, is used as the encoding of created strings.

`Encoding::default_internal` is initialized with -E option or nil otherwise.

    static VALUE
    get_default_internal(VALUE klass)
    {
        return rb_enc_default_internal();
    }

default_internal = enc or nil click to toggle source

Sets default internal encoding or removes default internal encoding when passed nil. You should not set `Encoding::default_internal` in ruby code as strings created before changing the value may have a different encoding from strings created after the change. Instead you should use `ruby -E` to invoke ruby with the correct default_internal.

See `Encoding::default_internal` for information on how the default internal encoding is used.

    static VALUE
    set_default_internal(VALUE klass, VALUE encoding)
    {
        rb_warning("setting Encoding.default_internal");
        rb_enc_set_default_internal(encoding);
        return encoding;
    }

find(string) → enc click to toggle source

Search the encoding with specified name. name should be a string.

    Encoding.find("US-ASCII")  #=> #<Encoding:US-ASCII>

Names which this method accept are encoding names and aliases including following special aliases

“external”

  default external encoding

“internal”

  default internal encoding

“locale”

  locale encoding

“filesystem”

  filesystem encoding

An `ArgumentError` is raised when no encoding with name. Only `Encoding.find("internal")` however returns nil when no encoding named “internal”, in other words, when Ruby has no default internal encoding.

    static VALUE
    enc_find(VALUE klass, VALUE enc)
    {
        int idx;
        if (is_obj_encoding(enc))
            return enc;
        idx = str_to_encindex(enc);
        if (idx == UNSPECIFIED_ENCODING) return Qnil;
        return rb_enc_from_encoding_index(idx);
    }

list → [enc1, enc2, ...] click to toggle source

Returns the list of loaded encodings.

    Encoding.list
    #=> [#<Encoding:ASCII-8BIT>, #<Encoding:UTF-8>,
          #<Encoding:ISO-2022-JP (dummy)>]

    Encoding.find("US-ASCII")
    #=> #<Encoding:US-ASCII>

    Encoding.list
    #=> [#<Encoding:ASCII-8BIT>, #<Encoding:UTF-8>,
          #<Encoding:US-ASCII>, #<Encoding:ISO-2022-JP (dummy)>]

    static VALUE
    enc_list(VALUE klass)
    {
        VALUE ary = rb_ary_new2(0);

        RB_VM_LOCK_ENTER();
        {
            rb_ary_replace(ary, rb_default_encoding_list);
            rb_ary_concat(ary, rb_additional_encoding_list);
        }
        RB_VM_LOCK_LEAVE();

        return ary;
    }

locale_charmap → string click to toggle source

Returns the locale charmap name. It returns nil if no appropriate information.

    Debian GNU/Linux
      LANG=C
        Encoding.locale_charmap  #=> "ANSI_X3.4-1968"
      LANG=ja_JP.EUC-JP
        Encoding.locale_charmap  #=> "EUC-JP"

    SunOS 5
      LANG=C
        Encoding.locale_charmap  #=> "646"
      LANG=ja
        Encoding.locale_charmap  #=> "eucJP"

The result is highly platform dependent. So `Encoding.find(Encoding.locale_charmap)` may cause an error. If you need some encoding object even for unknown locale, `Encoding.find`(“locale”) can be used.

    VALUE
    rb_locale_charmap(VALUE klass)
    {
    #if NO_LOCALE_CHARMAP
        return rb_usascii_str_new_cstr("US-ASCII");
    #else
        return locale_charmap(rb_usascii_str_new_cstr);
    #endif
    }

name_list → ["enc1", "enc2", ...] click to toggle source

Returns the list of available encoding names.

    Encoding.name_list
    #=> ["US-ASCII", "ASCII-8BIT", "UTF-8",
          "ISO-8859-1", "Shift_JIS", "EUC-JP",
          "Windows-31J",
          "BINARY", "CP932", "eucJP"]

    static VALUE
    rb_enc_name_list(VALUE klass)
    {
        VALUE ary;

        GLOBAL_ENC_TABLE_ENTER(enc_table);
        {
            ary = rb_ary_new2(enc_table->names->num_entries);
            st_foreach(enc_table->names, rb_enc_name_list_i, (st_data_t)ary);
        }
        GLOBAL_ENC_TABLE_LEAVE();

        return ary;
    }

### Public Instance Methods

ascii_compatible? → true or false click to toggle source

Returns whether ASCII-compatible or not.

    Encoding::UTF_8.ascii_compatible?     #=> true
    Encoding::UTF_16BE.ascii_compatible?  #=> false

    static VALUE
    enc_ascii_compatible_p(VALUE enc)
    {
        return RBOOL(rb_enc_asciicompat(must_encoding(enc)));
    }

dummy? → true or false click to toggle source

Returns true for dummy encodings. A dummy encoding is an encoding for which character handling is not properly implemented. It is used for stateful encodings.

    Encoding::ISO_2022_JP.dummy?       #=> true
    Encoding::UTF_8.dummy?             #=> false

    static VALUE
    enc_dummy_p(VALUE enc)
    {
        return RBOOL(ENC_DUMMY_P(must_encoding(enc)));
    }

inspect → string click to toggle source

Returns a string which represents the encoding for programmers.

    Encoding::UTF_8.inspect       #=> "#<Encoding:UTF-8>"
    Encoding::ISO_2022_JP.inspect #=> "#<Encoding:ISO-2022-JP (dummy)>"

    static VALUE
    enc_inspect(VALUE self)
    {
        rb_encoding *enc;

        if (!is_data_encoding(self)) {
            not_encoding(self);
        }
        if (!(enc = DATA_PTR(self)) || rb_enc_from_index(rb_enc_to_index(enc)) != enc) {
            rb_raise(rb_eTypeError, "broken Encoding");
        }
        return rb_enc_sprintf(rb_usascii_encoding(),
                              "#<%"PRIsVALUE":%s%s%s>", rb_obj_class(self),
                              rb_enc_name(enc),
                              (ENC_DUMMY_P(enc) ? " (dummy)" : ""),
                              rb_enc_autoload_p(enc) ? " (autoload)" : "");
    }

name → string

Returns the name of the encoding.

    Encoding::UTF_8.name      #=> "UTF-8"

Alias for: to_s
names → array click to toggle source

Returns the list of name and aliases of the encoding.

    Encoding::WINDOWS_31J.names  #=> ["Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK"]

    static VALUE
    enc_names(VALUE self)
    {
        VALUE args[2];

        args[0] = (VALUE)rb_to_encoding_index(self);
        args[1] = rb_ary_new2(0);

        GLOBAL_ENC_TABLE_EVAL(enc_table,
                              st_foreach(enc_table->names, enc_names_i, (st_data_t)args));

        return args[1];
    }

replicate(name) → encoding click to toggle source

Returns a replicated encoding of enc whose name is name. The new encoding should have the same byte structure of enc. If name is used by another encoding, raise `ArgumentError`.

    static VALUE
    enc_replicate_m(VALUE encoding, VALUE name)
    {
        int idx = rb_enc_replicate(name_for_encoding(&name), rb_to_encoding(encoding));
        RB_GC_GUARD(name);
        return rb_enc_from_encoding_index(idx);
    }

to_s → string click to toggle source

Returns the name of the encoding.

    Encoding::UTF_8.name      #=> "UTF-8"

    static VALUE
    enc_name(VALUE self)
    {
        return rb_fstring_cstr(rb_enc_name((rb_encoding*)DATA_PTR(self)));
    }

Also aliased as: name