# Class: Tokenizer

## ProtoBuf .DotProto. Tokenizer

prototype tokenizer

#### new Tokenizer(proto)

  Constructs a new Tokenizer.

  ##### Parameters:

  Name | Type | Description
  --- | --- | ---
  `proto` | string | Proto to tokenize

  Source:
    - ProtoBuf.js, line 430

### Members

#### index :number

  Current index.

  ##### Type:

  - number
  Source:
    - ProtoBuf.js, line 444

#### line :number

  Current line.

  ##### Type:

  - number
  Source:
    - ProtoBuf.js, line 451

#### readingString :boolean

  Whether currently reading a string or not.

  ##### Type:

  - boolean
  Source:
    - ProtoBuf.js, line 465

#### source :string

  Source to parse.

  ##### Type:

  - string
  Source:
    - ProtoBuf.js, line 437

#### stack :Array

  Stacked values.

  ##### Type:

  - Array
  Source:
    - ProtoBuf.js, line 458

#### stringEndsWith :string

  Whatever character ends the string. Either a single or double quote character.

  ##### Type:

  - string
  Source:
    - ProtoBuf.js, line 472

### Methods

#### next() → (nullable) {string}

  Gets the next token and advances by one.

  Source:
    - ProtoBuf.js, line 505

  ##### Throws:

  If it's not a valid proto file
  Type
    Error

  ##### Returns:

  Token or `null` on EOF

  Type
    string

#### peek() → (nullable) {string}

  Peeks for the next token.

  Source:
    - ProtoBuf.js, line 579

  ##### Throws:

  If it's not a valid proto file
  Type
    Error

  ##### Returns:

  Token or `null` on EOF

  Type
    string

#### toString() → {string}

  Returns a string representation of this object.

  Source:
    - ProtoBuf.js, line 594

  ##### Returns:

  String representation as of "Tokenizer(index/length)"

  Type
    string