lexical token
in lexical analysis, atomic string with an assigned and thus identified meaning; can be either word, bi-gram or sentence
en-GB/ˈlɛksɪkl ˈtəʊkən/
en-US/ˈlɛksɪkəl ˈtoʊkən/
in lexical analysis, atomic string with an assigned and thus identified meaning; can be either word, bi-gram or sentence