Nginx Source Code Notes – Configuration File Parsing

Understanding Nginx configuration files:

• Configuration directives have scopes, which are divided into global scopes and {}scopes created using `configure`.
• Different configuration directives within the same scope have no order of precedence; whether the same directive can be used within the same scope, and how to handle the same directive, is determined by each module itself.
• The behavior of all modules during the Nginx runtime is closely related to configuration directives.
• Each configuration directive can only be used within a predefined scope.
• Configuration command parsing uses a recursive approach.

The configuration parsing code is substantial, making this article inevitably lengthy. Therefore, please be prepared. Furthermore, since mail this module is rarely used in daily life, this article will only httpanalyze the configuration parsing process in specific application scenarios.

At the same time, for the sake of convenience, let’s agree on some names first:

• Configuration file – refers to Nginx’s main configuration file . The methods of importing other configuration files nginx.confwithin the configuration file will not be analyzed here.include
• Configuration directives – the basic unit in a configuration file. Configuration directives can accept zero or more configuration directive parameters .
• Configuration Items – Internal storage corresponding to configuration directives. The value of one configuration directive may affect the values ​​of multiple configuration items.
• Configuration Item Structure – A structure used for centralized management of configuration items related to each module.
• Scope module configuration arrayvoid * – An array of type `<module>` for each scope . Each pointer points to a configuration item structurevoid * for the module that can be used in this scope .

Basic Concepts

• Nginx modules ngx_module_tare represented using a structure.
• Nginx categorizes modules into types such as NGX_CORE_MODULE, NGX_EVENT_MODULE, NGX_HTTP_MODULE, and , based on their classification and level, and stores them in .NGX_MAIL_MODULEngx_module_t.type
• Nginx configuration directives ngx_command_tare “declared” using a structure. The configuration directives supported by the module ngx_module_t.commandsare stored in an array.
• The module context module contextdefines a series of operations for module configuration (requesting configuration storage, inheriting configuration from parent modules, etc.), and different operations are invoked at different stages of configuration analysis. Module contexts are categorized based on module type ngx_core_module_t, such as , ngx_event_module_t, ngx_http_module_t, ngx_mail_module_tetc. The module context is ngx_module_t.ctxstored in .
• The scope of configuration directives. Every configuration directive has a scope within which it can be used. If a configuration directive is used outside its scope, Nginx will print an error message and exit. The current scopes for configuration directives include NGX_MAIN_CONF` NGX_EVENT_CONF` NGX_HTTP_MAIN_CONF, NGX_HTTP_SRV_CONF` NGX_HTTP_LOC_CONF NGX_MAIL_MAIN_CONF`, NGX_MAIL_SRV_CONF…
  • A configuration directive can be defined in multiple scopes.
  • Instructions defined in a parent scope are inherited by child scopes. However, if a child scope also uses the same instruction, the instruction value in the child scope will override the instruction in the higher-level scope.
• The parameters of a configuration directive. A configuration directive can accept zero or more parameters. The number of parameters it can accept must be explicitly declared when defining the configuration directive so that Nginxthe correctness of the configuration file can be checked during the configuration parsing process.
  • There are some configuration directives, such as , http, event, server, location, mailetc. types, used to explicitly define the scope. They are marked as NGX_CONF_BLOCK, indicating {}the start of a block.
• Information such as the scope and acceptable parameters of configuration directives ngx_command_t.typeis stored in the field.
• Configuration command parsing function. Almost every configuration command (except for simple single-value commands) defines a corresponding callback function. NginxDuring configuration parsing, if the command is encountered, this callback function will be called to further parse the command.

The following diagram is a simple nginx.confillustration of scope:

• The internal structure of the Nginx configuration store is also organized by scope. Each scope reserves a space (one void *) for its supporting directives to appear in the modules within that scope.
• By default, the types of modules enabled by Nginx (modules listed in the order of activation from top to bottom) are as follows:

------------------------------------------------------
    module                          type
------------------------------------------------------
ngx_core_module                 NGX_CORE_MODULE
ngx_errlog_module               NGX_CORE_MODULE
ngx_conf_module                     NGX_CONF_MODULE
ngx_events_module               NGX_CORE_MODULE
ngx_event_core_module               NGX_EVENT_MODULE
ngx_epoll_module                    NGX_EVENT_MODULE
ngx_http_module                 NGX_CORE_MODULE
ngx_http_core_module                NGX_HTTP_MODULE
ngx_http_log_module                 NGX_HTTP_MODULE
ngx_http_upstream_module            NGX_HTTP_MODULE
ngx_http_autoindex_module           NGX_HTTP_MODULE
...                                 NGX_HTTP_MODULE

Preparation

There are two scenarios for configuration file parsing:

• When the Nginx process starts, it reads the configuration file and completes the corresponding initialization according to the configuration.
• When Nginx receives a signal to reload the configuration file, it reads the configuration file and performs the corresponding initialization based on the current configuration file. Then, it releases the resources allocated based on the old configuration file (except those reused by the new configuration).

This article only analyzes the first scenario.

Before configuration parsing begins, Nginx allocates a persistent memory pool and a temporary memory pool for the parsing process. The memory allocated from the persistent memory pool is used to store the configuration structure that has been parsed and correctly initialized from the configuration file; while the memory allocated from the temporary memory pool is only used for temporary storage during the parsing process and will be reclaimed after the configuration parsing is complete.

Nginx then prepares an array of module configuration indexes for the first level. NGX_CORE_MODULEThis first level only stores the module’s configuration structure, and its scope is NGX_MAIN_CONF:

    ------------core/ngx_cycle.c:183------------------------
    cycle->conf_ctx = ngx_pcalloc(pool, ngx_max_module * sizeof(void *));
    ...
    if (i = 0; ngx_modules[i]; i++) {
        if (ngx_modules[i]->type != NGX_CORE_MODULE) {
            continue;
        }

        module = ngx_modules[i]->ctx;
        if (module->create_conf) {
            rv = module->create_conf(cycle);
            ...
            cycle->conf_ctx[ngx_modules[i]->index] = rv;
        }
    }

Parsing framework

The configuration resolution process is quite simple and standard: prepare the current scope context, read the instruction and analyze the instruction parameters, and finally call the callback function corresponding to the instruction.

Taking the first-level NGX_MAIN_CONFscope as an example, its resolution process is roughly as follows:

    ------------core/ngx_cycle.c:246------------
    conf.ctx = cycle->conf_ctx;
    conf.cycle = cycle;
    conf.pool = pool;
    conf.log = log;
    conf.module_type = NGX_CORE_MODULE;
    conf.cmd_type = NGX_MAIN_CONF;
    ...
    ngx_conf_parse(&conf, &cycle->conf_file);

    for (i = 0; ngx_modules[i]; i++) {
        if (ngx_modules[i]->type != NGX_CORE_MODULE) {
            continue;
        }

        module = ngx_modules[i]->ctx;

        if (module->init_conf) {
            module->init_conf(cycle, cycle->conf_ctx[ngx_modules[i]->index]);
            ...
        }
    }

The code above parses NGX_CORE_MODULEmodules of type `<module>`, and the parsed instructions reside NGX_MAIN_CONFin the specified scope. Simultaneously, the parsed data is stored in `<module>` according to the module conf.ctx. After the configuration file is parsed, Nginx calls the `init( )` function NGX_CORE_MODULEof the `<module>` type init_confto perform further initialization operations based on the read configuration.

ngx_conf_parseThis is the entry point for configuration parsing. It reads the configuration file data based on the context, analyzes it, and checks the configuration file syntax. Additionally, this function is indirectly and recursively called after the instruction processing function modifies context information such as scope.

Let’s take a look at ngx_conf_parsethe execution process.

    ----------core/ngx_conf_file.c:101----------------
    char *
    ngx_conf_parse(ngx_conf_t *cf, ngx_str_t *filename)
    {
        ngx_buf_t buf;

        if (filename) {
            fd = ngx_open_file(filename->data, NGX_FILE_RDONLY, NGX_FILE_OPEN, 0);
            ...
            cf->conf_file = &conf_file;
            ...
            cf->conf_file->buffer = &buf;
            ...
            buf.start = ngx_alloc(NGX_CONF_BUFFER, cf->log);
            ...
            cf->conf_file->file.fd = fd;
            ...

            type = parse_file;

        } else if (cf->conf_file->file.fd != NGX_INVALID_FILE) {

            type = parse_block;

        } else {
            type = parse_param;
        ...

        for ( ;; ) {
            rc = ngx_conf_read_token(cf);
            ...
            rc = ngx_conf_handler(cf, rc);
        }
        ...
    }

The first ngx_conf_parsetime it’s called, the function opens the configuration file, sets the parsing type to be executed, reads it token, and then calls it ngx_conf_handler. So ngx_conf_handlerwhat does `f` actually do?

    -------------core/ngx_conf_file.c:279-------------
    static ngx_int_t
    ngx_conf_handler(ngx_conf_t *cf, ngx_int_t last)
    {
        ...
        for (i = 0; ngx_modules[i]; i++) {
            /* module type checking */
            ...
            cmd = ngx_modules[i]->commands;
            ...
            for ( /* void */ ; cmd->name.len; cmd++) {
                /* name comparison */
                ...
                /* namespace checking */
                ...
                /* checking argument numbers */
                ...
                /* set up the directive's configuration context */

                conf = NULL;

                if (cmd->type & NGX_DIRECT_CONF) {
                    conf = ((void **) cf->ctx)[ngx_modules[i]->index];

                } else if (cmd->type & NGX_MAIN_CONF) {
                    conf = &(((void **) cf->ctx)[ngx_modules[i]->index];

                } else if (cf->ctx) {
                    confp = *(void **) ((char *) cf->ctx + cmd->conf);

                    if (confp) {
                        conf = confp[ngx_modules[i]->ctx_index];
                    }
                }

                rc = cmd->set(cf, cmd, conf);
                ...
            }
        }
    }

A few additional points regarding the above code:

• After fully reading a configuration directive, Nginx searches for that directive name across all directive definitions. However, before performing the search, Nginx first verifies that the module type matches the type in the current parsing function context. Then, after finding a matching directive definition in a module, it verifies that the directive’s allowed scope includes the scope recorded in the current parsing function context. Finally, it checks that the number of parameters in the directive matches the number specified in the directive definition.
• After the verification is complete, Nginx compares the directive name with all predefined supported directives in all modules to find a complete match. The storage location of the configuration item varies depending on the type of the directive.
  • NGX_DIRECT_CONFConfiguration directives of this type have configuration items stored in the global scope. This type of directive primarily appears in ngx_core_modulemodules.conf = ((void **) cf->ctx)[ngx_modules[i]->index];
  • NGX_MAIN_CONFThis indicates that the scope of the configuration directive is the global scope. Looking at the entire Nginx code, besides ngx_core_modulethe configuration directive `<configuration_scope>` (also marked as `<scope> NGX_DIRECT_CONF`) which resides in this scope, the other directives defining new child scopes—`<scope>` , `<scope>`, `<scope>`, and `<scope>` events—are httpnot mailallocated space in the global scope. Therefore, the space allocated in the directive’s handler function needs to be attached to the global scope.imapNGX_DIRECT_CONFNGX_MAIN_CONFconf = &(((void **) cf->ctx)[ngx_modules[i]->index];
  • The storage location of other types of configuration directives and the scope in which the directives appear (excluding the global scope) are related:confp = (void ) ((char ) cf->ctx + cmd->conf);if (confp) { conf = confp[ngx_modules[i]->ctx_index]; }
• Once the location where the configuration items will be stored is determined, the instruction callback function is called to complete the configuration item initialization and other tasks.rc = cmd->set(cf, cmd, conf);

This is roughly the framework for parsing configuration files. Before delving into a more detailed code analysis, let’s first look at a diagram of the configuration structure after parsing the configuration file:

Instruction Reading

Having grasped the general framework, let’s now delve into the details. Next, we’ll analyze the configuration command reading and decomposition functions, that is, ngx_conf_read_tokenthe function implementation.

ngx_conf_read_tokenThe system reads partial data from the disk configuration file into memory buffer, and then bufferreads a valid instruction from it in a format supported by Nginx. If bufferthe data in the configuration file is insufficient to constitute a complete instruction, more data needs to be read from the configuration file.

configuration file reading

bufferWhen there is insufficient data in the configuration file, that is b->pos >= b->last, when the configuration file is empty, more data is read from the configuration file.

    --------------core/ngx_conf_file.c:463---------------
    if (b->pos >= b->last) {
        ...
        len = b->pos - start;

        if (len) {
            ngx_memcpy(b->start, start, len);
        }

        size = (ssize_t) (file_size - cf->conf_file->file.offset);

        if (size > b->end - (b->start + len)) {
            size = b->end - (b->start + len);
        }

        n = ngx_read_file(&cf->conf_file->file, b->start + len, size,
                          cf->conf_file->file.offset);
        b->pos = b->start + len;
        b->last = b->pos + n;
        start = b->start;
    }

Additional notes on the above code:

• ngx_buf_tHere’s a brief description of its structure and fields: ———core/ngx_buf.h:19—————– struct ngx_buf_s { u_char pos; / Start position of unused data in b / u_char last; / End position of valid data stored in b / … u_char start; / Start position of contiguous memory block b / u_char end; / End position of contiguous memory block b / … }; typedef struct ngx_buf_s ngx_buf_t;
• bufferWhen all valid data in the instruction has been used up b->pos >= b->last, a certain component of an instruction (from startthe start to b->posthe end lenbytes) may not have been fully read, and this part of the data needs to be retained.ngx_memcpy(b->start, start, len);
• bufferNow b->end - (b->start + len)the available space has been freed up.
• startIt always points to the beginning of a component of an instruction.

Syntax state machine

ngx_conf_read_tokenThe remaining part implements the state machine for character-by-character parsing of the instruction line syntax. This part of the code is located in [location missing core/ngx_buf.h:593-734]. Semantic correctness verification is done in the base part of the configuration parsing.

• The allowed format for commands to appear on the line (using regular expression definitions; I’m not familiar with the syntax, so please bear with me / TODO /):line := token token? { line+ } | token space token? space?; token := ["'][ \t\r\na-z0-9\\$]+["'] | [a-z0-9$]+ | ( token ) space := [ \t\r\n]+
• last_spaceUsed to indicate whether the previous character is true or false space. Initial value is…1
• need_spaceUsed to indicate that the state machine requires the next character to be space(or ‘;’, ‘{‘, ‘)’). Initial value0
• startPoints to the starting position of the analysis token. The initial value is…b->pos
• d_quotedUsed to indicate that the current character is in ""the range. Initial value is…0
• s_quotedUsed to indicate that the current character is in ''the range. Initial value is…0
• sharp_commentUsed to indicate that the current character is part of a comment. Initial value is…0
• variableUsed to indicate that the current character is part of the variable name. Initial value is…0
• quoteUsed to indicate whether the character preceding the current character is zero \. Initial value is…0
• cf->argsStores the instruction name and all parameters. It is ngx_str_tan array of type `int`.

This part of the code is too long (over 200 lines), so only the framework is summarized below.

    for ( ;; ) {
        ch = *b->pos++; /* 当前字符存储于 ch 中 */

        if (ch == LF) {
            if (sharp_comment) {  /* # 是行注释 */
                sharp_comment = 0;
            }
        }

        if (sharp_comment) {    /* 忽略 # 后的所有字符，直到进行新行为止 */
            continue;
        }

        if (quoted) {           /* 不再单独处理 \ 后的字符，后面读取 token
            quoted = 0;            时，会做专门处理 */
            continue;
        }

        if (need_space) {       /* 此字符必须为 space 分隔符 */
            /*
              忽略掉 space 字符；
              如果字符是 ';'，'{'，此次指令行读取完成；
              如果字符是 ')'，开始识别新的 token；
              如果读到其它字符则解析失败
             */
        }

        if (last_space) { /* 找到新 token 的开始位置 - 第一个非 space 字符 */
            /*
              忽略掉 space 字符；
              如果字符是 ';''{''}'，此次指令行读取完成；
              如果字符是 '#'，该字符后面都是注释；
              如果字符是 '\'，下一个字符将被忽略；
              如果字符是 '"', ''', 下一个字符是被引用的内容；
             */
        } else { /* 开始读取新 token */
            /*
              如果字符是 '$'，后面的非 space 字符组成变量 token；
              碰到了结束引号 '"', ''' 时，token 读取完成；
              碰到了 space 字符，token 读取完成；
             */
            if (found) {
                /*
                  将找到的 token 追加到 cf->args 数组中，并且每个 token 字符
                  串以 '\0' 结束
                 */
            }
        }
    }

Scope resolution

eventsScope

eventsThe scope is created by ngx_events_modulethe eventsinstruction handling of `__init__`. It is defined as follows:

    -------------event/ngx_event.c:83------------
    { ngx_string("events),
      NGX_MAIN_CONF|NGX_CONF_BLOCK|NGX_CONF_NOARGS,
      ngx_events_block,
      0,
      0,
      NULL }

Before the configuration file parsing begins, ngx_events_modulethe structure pointer is set to NULL in the global scope. Therefore, ngx_events_blockthe first thing the function does is create the configuration structure and mount it to the global scope.

    ------------event/ngx_event.c:902------------
    ctx = ngx_pcalloc(cf->pool, sizeof(void *));
    ...
    *ctx = ngx_pcalloc(cf->pool, ngx_event_max_module * sizeof(void *));
    ...
    *(void **) conf = ctx;

Additional notes on the above code:

• Why go through the extra step of using a pointer ctxto void *a type that then points to the scoped module configuration array? Read on!

Then, the function NGX_EVENT_MODULEin the module context of type `<module> create_conf` is called to allocate a configuration resolver for each module in this scope. Taking `<module>` ngx_epoll_moduleas an example, its create_conffunction requests ngx_epoll_conf_ta memory block from the memory pool to store `<module>` and attaches it to eventsthe corresponding location in the `<module>` scope.

Then, as described above, the configuration resolution context is switched, and then the function is called to resolve the command lines appearing in ngx_conf_parsethe configuration file corresponding to this scope :block

    -----------event/ngx_event.c:929-------------
    pcf = *cf;
    cf->ctx = ctx;
    cf->module_type = NGX_EVENT_MODULE;
    cf->cmd_type = NGX_EVENT_CONF;

    rv = ngx_conf_parse(cf, NULL);

    *cf = pcf;

Let’s take the module as an example to see how the whole process works when epollit encounters the instructions it supports in the above context .epoll_events

    ----------event/modules/ngx_epoll_module.c:129------------
    { ngx_string("epoll_events"),
      NGX_EVENT_CONF|NGX_CONF_TAKE1,
      ngx_conf_set_num_slot,
      0,                                         /* conf */
      offsetof(ngx_epoll_conf_t, events),        /* offset */
      NULL },

After ngx_conf_read_tokena successful return, ngx_conf_handlerthe location of the configuration item corresponding to the instruction is determined based on the instruction type and context:

    ----------core/ngx_conf_file.c:386------------------------
    confp = *(void **) ((char *) cf->ctx + cmd->conf)

    if (confp) {
        conf = confp[ngx_modules[i]->ctx_index];
    }

Additional notes on the above code:

• cf->ctxngx_events_blockThe pointer points to the allocated value in ` <scope>` void *, void *which in turn points to eventsthe scope module configuration array. Therefore, confpultimately, the pointer points to eventsthe scope module configuration array, confwhich in turn points to ngx_epoll_modulethe corresponding configuration item structure ngx_epoll_conf_t.

epoll_eventsThe instruction processing function is a generic processing function ngx_conf_set_num_slot.

    ------------core/ngx_conf_file.c:1182-------------------
    char *
    ngx_conf_set_num_slot(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
    {
        char *p = conf;
        ...
        np = (ngx_int_t *) (p + cmd->offset);
        ...

        *np = ngx_atoi(value[1].data, value[1].len);
    }

Additional notes on the above code:

• cmd->offsetThe value is the offset of the configuration item corresponding to the configuration offsetof(ngx_epoll_conf_t, events)directive within the configuration structure . Nginx predefined processing functions generally use this approach to achieve versatility.epoll_eventsngx_uint_t eventsngx_epoll_conf_t

Ultimately, epoll_eventsafter the above process, the command parameters are stored in the configuration structure and members within eventsthe scope .ngx_epoll_modulengx_epoll_conf_tevents

eventsThe parsing process for other instructions within the scope epoll_eventsis largely the same, so we’ll skip that. Let’s now discuss httpscope.

httpScope

httpThe scope is created by ngx_http_modulethe httpinstruction handler function. It is defined as follows:

    -------------event/ngx_event.c:83------------
    { ngx_string("http),
      NGX_MAIN_CONF|NGX_CONF_BLOCK|NGX_CONF_NOARGS,
      ngx_http_block,
      0,
      0,
      NULL }

Similar to eventsscope, before the configuration file is parsed, ngx_http_modulethe structure pointer is set to NULL in the global scope. Therefore, ngx_http_blockthe first thing the function does is also to create the configuration structure and attach it to the global scope.

    ------------http/ngx_http.c:128--------------
    ctx = ngx_pcalloc(cf->pool, sizeof(ngx_http_conf_ctx_t));
    ...
    *(ngx_http_conf_ctx_t **) conf = ctx;

However, httpthe scope and eventsthe place of scope ctxdo not point to a single entity void *, but rather to a ngx_http_conf_ctx_tstructure.

    ------------http/ngx_http_config.h:16--------
    typedef struct {
        void        **main_conf;
        void        **srv_conf;
        void        **loc_conf;
    } ngx_http_conf_ctx_t;

The reason for this difference is:

• httpScopes can be nested. A httpsingle scope can contain multiple nested serverscopes, and a single serverscope can also contain multiple nested locationscopes.
• Furthermore, NGX_HTTP_MODULEdirectives for a module type can be defined in httpthe `main` scope NGX_HTTP_MAIN_CONF, serverthe `external` scope NGX_HTTP_SRV_CONF, or locationthe `extend` scope NGX_HTTP_LOC_CONF. In fact, most HTTP module directives are defined in all three scopes simultaneously.
• NGX_HTTP_MODULEThe module categorizes its supported instructions into three types of configuration items: the first type of instructions can only NGX_HTTP_MAIN_CONFbe used in the `<input>` scope; the second type can be used in the `<output>` and `<output>` scopes; and the third type can be used in the NGX_HTTP_MAIN_CONF `<output> `, `<output> `, and `<output>` scopes. The configuration items for these three types of instructions are stored in three structures : `<struct> `, `<struct>` , and ` <output>` .NGX_HTTP_SRV_CONFNGX_HTTP_MAIN_CONFNGX_HTTP_SRV_CONFNGX_HTTP_LOC_CONFngx_http_xxx_main_conf_tngx_http_xxx_srv_conf_tngx_http_xxx_loc_conf_t
• main_conf`, srv_conf` and `, loc_conf` are the scoped module configuration arrays provided for the three types of configuration structures. In httpthe `local` scope, all three arrays are required; in the server`local` scope and its parent scope , the http same array is used; main_confin the ` locationlocal` scope, serverthe same array is used as well as its parent scope .main_confsrv_conf

After allocating the module configuration arrays for the three scopes in ngx_http_block`<module> `, the callback functions defined by the module context for each type are called to create the three types of configuration item structures for the module in the three scope module configuration arrays respectively:httpNGX_HTTP_MODULEcreate_main_confcreate_srv_confcreate_loc_conf

    -------------http/ngx_http.c:150---------------
    ctx->main_conf = ngx_pcalloc(cf->pool, sizeof(void *) * ngx_http_max_module);
    ...
    ctx->srv_conf = ngx_pcalloc(cf->pool, sizeof(void *) * ngx_http_max_module);
    ...
    ctx->loc_conf = ngx_pcalloc(cf->pool, sizeof(void *) * ngx_http_max_module);
    ...
    for (m = 0; ngx_modules[m]; m++) {
        if (ngx_modules[m]->type != NGX_HTTP_MODULE) {
            continue;
        }

        module = ngx_modules[m]->ctx;
        mi = ngx_modules[m]->ctx_index;

        if (module->create_main_conf) {
            ctx->main_conf[mi] = module->create_main_conf(cf);
            ...
        }

        if (module->create_srv_conf) {
            ctx->srv_conf[mi] = module->create_srv_conf(cf);
            ...
        }

        if (module->create_loc_conf) {
            ctx->loc_conf[mi] = module->create_loc_conf(cf);
            ...
        }
     }

Next, eventssimilar to the scope, switch the configuration resolution context and call ngx_conf_parsethe function:

    -------------http/ngx_http.c:214------------
    pcf = *cf;
    cf->ctx = ctx;
    ...
    cf->module_type = NGX_HTTP_MODULE;
    cf->cmd_type = NGX_HTTP_MAIN_CONF;
    rv = ngx_conf_parse(cf, NULL);
    ...
    *cf = pcf;

httpThe resolution of ordinary instructions within a scope is largely the same as the resolution process eventsdescribed above .ngx_epoll_moduleepoll_events

The following analysis will focus on the creation of a scope httpwithin a scope server, the creation of serversub-scopes , and how multiple scopes are attached to their parent scope .locationlocationserverserverhttp

serverScope

The configuration resolution function is now in httpthe context of the scope: * cf->ctxpoints to ngx_http_conf_ctx_t * cf->module_typevalues ​​to NGX_HTTP_MODULE * cf->cmd_typevaluesNGX_HTTP_MAIN_CONF

When the parsing function encounters an instruction server {}, it will call serverthe instruction’s handler function ngx_http_core_server.

    -------------http/ngx_http_core_module.c:2300------------
    static char *
    ngx_http_core_server(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy)
    {
        ctx = ngx_pcalloc(cf->pool, sizeof(ngx_http_conf_ctx_t));
        ...
        http_ctx = cf->ctx;
        ctx->main_conf = http_ctx->main_conf;

        ctx->srv_conf = ngx_pcalloc(cf->pool, sizeof(void *) * ngx_http_max_moudle);
        ...
        ctx->loc_conf = ngx_pcalloc(cf->pool, sizeof(void *) * ngx_http_max_module);
        ...
        cscf = ctx->srv_conf[ngx_http_core_module.ctx_index];
        cscf->ctx = ctx;

        cmcf = ctx->main_conf[ngx_http_core_module.ctx_index];

        cscfp = ngx_array_push(&cmcf->servers);
        ...
        *cscfp = cscf;

        pcf = *cf;
        cf->ctx = ctx;
        cf->cmd_type = NGX_HTTP_SRV_CONF;

        rv = ngx_conf_parse(cf, NULL);

        *cf = pcf;
    }

Additional notes on the above code:

• serverThe scope and its parent scope share a common key.main_conf
• serverThe scope also needs to use ngx_http_conf_ctx_tan array of three types of scope module configurations representing this scope.
• serverThe scope ngx_http_conf_ctx_tis stored in the ngx_http_core_moduleconfiguration item structure ngx_http_core_srv_conf_t.ctx
• ngx_http_core_modulehttpIt is responsible for associating scopes within the scope and its various child scopes. serverThe scope that has just been resolved will be stored ( ngx_http_core_srv_conf_tindirectly stored) in the array ngx_http_core_main_conf_tof serversscopes.

Switch context before the configuration function is called again:

    pcf = *cf;
    cf->ctx = ctx;
    cf->cmd_type = NGX_HTTP_SRV_CONF;

    rv = ngx_conf_parse(cf, NULL);

    *cf= pcf;

Below, we will take listenthe example of instruction execution to analyze the instruction parsing process in the current scope.

listenThe instruction is defined as follows:

    --------------http/ngx_http_core_module.c:278---------
    { ngx_string("listen"),
      NGX_HTTP_SRV_CONF|NGX_CONF_1MORE,
      ngx_http_core_listen,
      NGX_HTTP_SRV_CONF_OFFSET,
      0,
      NULL },

ngx_conf_read_tokenAfter reading this instruction from the configuration file, ngx_conf_handlerits validity is verified, and then the storage location of the structure containing the instruction item is found.

confp = *(void **) ((char *) cf->ctx + cmd->conf);

    if (confp) {
        conf = confp[ngx_modules[i]->ctx_index];
    }

Additional notes on the above code:

• cf->ctxNow it points to serverthe value allocated after entering the scope ngx_http_conf_ctx_t. cmd->confThe value is NGX_HTTP_SRV_CONF_OFFSET, that is offsetof(ngx_http_conf_ctx_t, srv_conf), . Therefore, confpit ultimately points to the newly allocated ngx_http_conf_ctx_tscope srv_confmodule configuration array.
• confIt points to the serverscope .ngx_http_core_modulengx_http_core_srv_conf_t

Then, listenthe instruction processing function is called ngx_http_core_listen:

    -------------http/ngx_http_core_module.c:3256--------
    static char *
    ngx_http_core_listen(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
    {
        ngx_http_core_srv_conf_t *cscf = conf;
        ...
        cscf->listen = 1;
        ...
        ngx_http_add_listen(cf, cscf, &lsopt);
        ...
    }

    ------------http/ngx_http.c:1105-----------------
    ngx_int_t
    ngx_http_add_listen(ngx_conf_t *cf, ngx_http_core_srv_conf_t *cscf,
        ngx_http_listen_opt_t *lsopt)
    {
        ...
        ngx_http_conf_port_t        *port;
        ngx_http_core_main_conf_t   *cmcf;
        ...
        cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module);
        ...
        port = ngx_array_push(cmcf->ports);
        ...
        return ngx_http_add_address(cf, cscf, port, lsopt);
    }

    -----------http/ngx_http.c:1281-----------------
    static ngx_int_t
    ngx_http_add_address(ngx_conf_t *cf, ngx_http_core_srv_conf_t *cscf,
        ngx_http_conf_port_t *port, ngx_http_listen_opt_t *lsopt)
    {
        ...
        addr = ngx_array_push(&port->addrs);
        ...
        return ngx_http_add_server(cf, cscf, addr);
    }

    -----------http/ngx_http.c:1319--------------------
    static ngx_int_t
    ngx_http_add_server(ngx_conf_t *cf, ngx_http_core_srv_conf_t *cscf,
        ngx_http_conf_addr_t *addr)
    {
        ...
        server = ngx_array_push(&addr->servers);
        ...
        *server = cscf;
        ...
    }

listenThe command configuration is thus parsed and stored. Finally, a data structure diagram of the Nginx listening address and port is attached.

locationScope

The configuration resolution function is now serverin the context of the scope: * cf->ctxpoints to * values server​​created using the scope, * values *ngx_http_conf_ctx_tcf->module_typeNGX_HTTP_MODULEcf->cmd_typeNGX_HTTP_SRV_CONF

When the parsing function encounters an instruction location {}, it will call locationthe instruction’s handler function ngx_http_core_location.

    -------------http/ngx_http_core_module.c:271-----------
    { ngx_string("location"),
      NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_BLOCK|NGX_CONF_TAKE12,
      ngx_http_core_location,
      NGX_HTTP_SRV_CONF_OFFSET,
      0,
      NULL },

Returning to the previous section, the parameters of the instruction processing function ngx_conf_handlerare calculated from the current context :location

    confp = *(void **) ((char *) cf->ctx + cmd->conf);

    if (confp) {
        conf = confp[ngx_modules[i]->ctx_index];
    }

Additional notes on the above code:

• confpIt points to the variable serverallocated in the parent scope .ngx_http_conf_ctx_tsrv_conf
• confIt points to ngx_http_core_modulethe ngx_http_core_srv_conf_tconfiguration structure.
• confThe parameter ngx_http_core_locationwas not used.

Next, ngx_http_core_locationthe function was called.

    -----------http/ngx_http_core_module.c:2391-----------
    static char *
    ngx_http_core_location(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy)
    {
        ...
        ctx = ngx_pcalloc(cf->pool, sizeof(ngx_http_conf_ctx_t));
        ...
        pctx = cf->ctx;
        ctx->main_conf = pctx->main_conf;
        ctx->srv_conf = pctx->srv_conf;

        ctx->loc_conf = ngx_pcalloc(cf->pool, sizeof(void *) * ngx_http_max_module);
        ...
        clcf = ctx->loc_conf[ngx_http_core_module.ctx_index];
        clcf->loc_conf = ctx->loc_conf;
        ...
        pclcf = pctx->loc_conf[ngx_http_core_module.ctx_index];
        ...

        ngx_http_add_location(cf, &pclcf->locations, clcf);

        save = *cf;
        cf->ctx = ctx;
        cf->cmd_type = NGX_HTT_LOC_CONF;

        rc = ngx_conf_parse(cf, NULL);

        *cf = save
        ...
    }

The relationships established by the code above can be found in the full configuration structure diagram provided in this article.

The parsing of other common instructions is similar to the process described above, so I won’t go into details.

At this point, the entire httpscope configuration file parsing process is complete. After the configuration file is parsed, ngx_http_blocka series of other tasks will be performed, such as initializing the three types of configuration structures for all modules, ensuring that the innermost scope inherits the same directive value from the outer scope, reorganizing locationand initializing phase, and initializing the HTTP request headers allowed by Nginx, etc.

Scope inheritance

As previously mentioned, httpscopes can be nested. Furthermore, NGX_HTTP_MODULEmost directives from a type module can be used simultaneously in both httpthe parent and child scopes (`<inner scope> server`, ` <child locationscope>`). Therefore, inner child scopes need to inherit configuration items from their parent scope.

The ultimate result of this behavior is that NGX_HTTP_MAIN_CONFdirective configuration items appearing in `<parent>` will still take effect in ` NGX_HTTP_SRV_CONF<child>` and NGX_HTTP_LOC_CONF`<child>` (when the directive is not explicitly configured in the child scope). Furthermore, configuration items in child scopes can override configuration items in parent scopes.

This goal was ngx_http_blockachieved by the function. Before proceeding, let’s bring our thoughts back. Note the current configuration resolution context:

* `cf->ctx` 指向 `NGX_HTTP_MAIN_CONF` 作用域的 `ngx_http_conf_ctx_t`
* `cf->module_type` 值为 `NGX_HTTP_MODULE`
* `cf->cmd_type` 值为 `NGX_HTTP_MAIN_CONF`

Therefore, after parsing the instructions of the entire httpscope,

    ------------http/ngx_http.c:246-------------
    cmcf = ctx->main_conf[ngx_http_core_module.ctx_index];
    cscfp = cmcf->servers.elts;

    for (m = 0; ngx_modules[m]; m++) {
        if (ngx_modules[m]->type != NGX_HTTP_MODULE) {
            continue;
        }

        module = ngx_modules[m]->ctx;
        mi = ngx_modules[m]->ctx_index;

        if (module->init_main_conf) {
            rv = module->init_main_conf(cf, ctx->main_conf[mi]);
            ...
        }

        for (s = 0; s < cmcf->servers.nelts; s++) {
            if (module->merge_srv_conf) {
                rv = module->merge_srv_conf(cf, ctx->srv_conf[mi],
                                            cscfp[s->ctx->srv_conf[mi]);
                ...
            }

            if (module->merge_loc_conf) {
                rv = module->merge_loc_conf(cf, ctx->loc_conf[mi],
                                            cscfp[s]->ctx->loc_conf[mi]);
                ...
                clcf = cscfp[s]->ctx->loc_conf[ngx_http_core_module.ctx_index];

                rv = ngx_http_merge_locations(cf, clcf->locations,
                                              cscfp[s]->ctx->loc_conf,
                                              module, mi);
            }
        }
    }

Additional notes on the above code:

• First, refer to the structure diagram above.
• The code only main_confinitializes the configuration structure for each module in the scope’s module configuration array. This is because these structures are used at runtime. The configuration structures in `<module_name>` srv_confand loc_conf`<module_name>` are not directly used at runtime. If some module configuration items are assigned values ​​during resolution, the subsequent merge operation will copy these assigned configuration items to their corresponding positions in the child scopes.
• mergeThe operation process is as follows: if a configuration item in a child scope is not assigned a value during the resolution process, the same configuration item value from the parent scope is copied to this configuration item; if a configuration item in a child scope is assigned a value during the resolution process, it is left as is; if neither the child scope configuration item nor the parent scope configuration item is initialized, the default value preset in the code is filled in.

The following ngx_http_core_modulecode snippets of relevant functions are used to verify the above conclusions:

    ----------------http/ngx_http_core_module.c:2751-------------
    static char *
    ngx_http_core_init_main_conf(ngx_conf_t *cf, void *conf)
    {
        ngx_http_core_main_conf_t *cmcf = conf;

        if (cmcf->server_names_hash_max_size == NGX_CONF_UNSET_UINT) {
            cmcf->server_names_hash_max_size = 512;
        }
        ...
    }

    ---------------http/ngx_http_core_module.c:2821-------------
    static char *
    ngx_http_core_merge_srv_conf(ngx_conf_t 8cf, void *parent, void *child)
    {
        ngx_http_core_srv_conf_t *prev = parent;
        ngx_http_core_srv_conf_t *conf = child;
        ...
        ngx_conf_merge_size_value(conf->connection_pool_size,
                                  prev->connection_pool_siez, 256);
        ...
    }

    #define ngx_conf_merge_size_value(conf, prev, default)          \
        if (conf == NGX_CONF_UNSET_SIZE) {                          \
            conf = (prev == NGX_CONF_UNSET_SIZE) ? default : prev;  \
        }

That concludes the explanation of how scope inheritance is implemented.

Final sorting

As mentioned above, httpafter the scope resolution is completed, ngx_http_blocka series of initialization tasks will continue to be performed, such as organizing locationthe configuration into a structure that can be accessed quickly, initializing the request header structure, initializing phasethe access structure, etc.

These topics could all be analyzed in their own separate articles, so I’ll skip them for now.

The request has arrived.

When a HTTPrequest arrives at Nginx, at what stage should the configuration scope be used, and how is the configuration obtained?

The requestinitialization function will be initialized once ngx_http_init_requestby a newly created configuration context (at this point, it is still impossible to determine which scope the request belongs to, so the default scope is used for the time being):requestserverserver

    ---------------http/ngx_http_request.c:384------------
    cscf = addr_conf->default_server;

    r->main_conf = cscf->ctx->main_conf;
    r->src_conf = cscf->ctx->srv_conf;
    r->loc_conf = cscf->ctx->loc_conf;
    ...

After reading HTTPthe request header and determining which serverscope should handle the request, requestthe configuration context needs to be adjusted as follows:

    -------------http/ngx_http_request.c:1688-------------
    static ngx_int_t
    ngx_http_find_virtual_server(ngx_http_request_t *r, u_char *host, size_t len)
    {
        ...
        r->srv_conf = cscf->ctx->srv_conf;
        r->loc_conf = cscf->ctx->loc_conf;
        ...
    }

At this point, once the scope to which r->loc_confthe pointer belongs is determined , the pointer will then be adjusted .serverloc_confrequestlocationr->loc_conf

At the same time, internally request, both rewritten requestand redirected functions requestneed to change r->loc_confthe pointer of the same function to achieve the function of jumping requestbetween different scopes.location

In addition, the configuration structure is mainly obtained through the following macros:

    #define ngx_http_get_module_main_conf(r, module)                             \
        (r)->main_conf[module.ctx_index]
    #define ngx_http_get_module_srv_conf(r, module)  (r)->srv_conf[module.ctx_index]
    #define ngx_http_get_module_loc_conf(r, module)  (r)->loc_conf[module.ctx_index]


    #define ngx_http_conf_get_module_main_conf(cf, module)                        \
        ((ngx_http_conf_ctx_t *) cf->ctx)->main_conf[module.ctx_index]
    #define ngx_http_conf_get_module_srv_conf(cf, module)                         \
        ((ngx_http_conf_ctx_t *) cf->ctx)->srv_conf[module.ctx_index]
    #define ngx_http_conf_get_module_loc_conf(cf, module)                         \
        ((ngx_http_conf_ctx_t *) cf->ctx)->loc_conf[module.ctx_index]

    #define ngx_get_conf(conf_ctx, module)  conf_ctx[module.index]

Don’t leave me so easily, please leave something behind…