git404hub

what is lexer fr?

doctrine/lexer — explained in plain English

Analysis updated 2026-06-24

11,166PHPAudience · developerComplexity · 2/5Setup · moderate

tl;dr

A PHP base class for building lexers, components that read raw text and break it into labeled tokens as the first step in parsing a custom language or query syntax.

vibe map

mindmap
  root((repo))
    What it does
      Base lexer class
      Token reading
      Parser foundation
    How it works
      Extend base class
      Define token types
      Recursive descent
    Used by
      Doctrine ORM DQL
      Doctrine Annotations
    Audience
      PHP developers
      Parser builders

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Extend the base lexer class to tokenize a custom query language or configuration file format in PHP.

VIBE 2

Use it as a foundation for a recursive descent parser that processes structured text input.

VIBE 3

Add Doctrine ORM to a PHP project and benefit from this library's tokenization indirectly for Doctrine Query Language.

what's the stack?

PHP

how it stacks up fr

doctrine/lexerz-song/laravel-adminoctobercms/october
Stars11,16611,15411,136
LanguagePHPPHPPHP
Setup difficultymoderateeasymoderate
Complexity2/52/53/5
Audiencedeveloperdeveloperdeveloper

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · moderate time til it works · 1h+

Documentation is minimal in the README, study Doctrine ORM or Annotations source code for real usage examples.

in plain english

Doctrine Lexer is a PHP library that provides a base class for building a lexer. A lexer is a program component that reads raw text and breaks it into a sequence of labeled tokens, which is the first step in parsing a custom language or query syntax. For example, if you had a query like "SELECT name FROM users WHERE age > 30", a lexer would identify each word and symbol as a specific type of token before any other logic tries to understand what the query means. This library is described as a base for use in top-down, recursive descent parsers. That refers to a common style of parser design where the code processes tokens from left to right and builds up an understanding of the input by calling itself recursively. The library does not implement a full parser but gives you the foundational tools to build one. The README is short and says this lexer is used internally by two other Doctrine projects: Doctrine Annotations (a PHP annotation parsing library) and Doctrine ORM, specifically for its Doctrine Query Language (DQL). Doctrine is a well-known set of PHP database and object persistence libraries. The high star count on this repository reflects how many PHP projects depend on Doctrine ORM and therefore pull in this library indirectly. There is no detailed usage documentation in the README itself. Developers who need to use or extend this library are pointed to the project page at doctrine-project.org. Given how brief the repository description is, it is clearly intended for developers already working within the Doctrine ecosystem rather than newcomers. The high star count reflects indirect popularity, since many PHP applications that use Doctrine ORM automatically pull in this library as a transitive dependency.

prompts (copy fr)

prompt 1
I'm building a PHP parser for a custom query syntax. How do I extend Doctrine Lexer's base class to tokenize expressions like [example syntax]?
prompt 2
Show me how to define token types and patterns in Doctrine Lexer so I can parse [describe format].
prompt 3
Write a recursive descent parser in PHP using Doctrine Lexer that handles simple arithmetic like 2 + 3 * 4.
prompt 4
Explain how Doctrine ORM uses doctrine/lexer to parse DQL queries, and how I can use the same pattern for my own query language.

Frequently asked questions

what is lexer fr?

A PHP base class for building lexers, components that read raw text and break it into labeled tokens as the first step in parsing a custom language or query syntax.

What language is lexer written in?

Mainly PHP. The stack also includes PHP.

How hard is lexer to set up?

Setup difficulty is rated moderate, with roughly 1h+ to a first successful run.

Who is lexer for?

Mainly developer.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.