Click here to Skip to main content
15,895,557 members
Home / Discussions / C / C++ / MFC
   

C / C++ / MFC

 
GeneralRe: File IO library Pin
Calin Negru26-Jun-22 11:17
Calin Negru26-Jun-22 11:17 
AnswerRe: File IO library Pin
k505426-Jun-22 4:21
mvek505426-Jun-22 4:21 
AnswerRe: File IO library Pin
Richard MacCutchan26-Jun-22 4:49
mveRichard MacCutchan26-Jun-22 4:49 
Questionhello guy my name is SARORN can i ask you about any inform? Pin
Sa Ron 202224-Jun-22 5:40
Sa Ron 202224-Jun-22 5:40 
AnswerRe: hello guy my name is SARORN can i ask you about any inform? Pin
Victor Nijegorodov24-Jun-22 5:51
Victor Nijegorodov24-Jun-22 5:51 
AnswerRe: hello guy my name is SARORN can i ask you about any inform? Pin
k505424-Jun-22 6:27
mvek505424-Jun-22 6:27 
AnswerRe: hello guy my name is SARORN can i ask you about any inform? Pin
RedDk24-Jun-22 8:42
RedDk24-Jun-22 8:42 
Questionc++ Pin
tuanguyen200021-Jun-22 22:59
tuanguyen200021-Jun-22 22:59 
QuestionRe: c++ Pin
CPallini22-Jun-22 0:02
mveCPallini22-Jun-22 0:02 
Questionc++ Pin
tuanguyen200021-Jun-22 6:49
tuanguyen200021-Jun-22 6:49 
AnswerRe: c++ Pin
tuanguyen200021-Jun-22 6:50
tuanguyen200021-Jun-22 6:50 
GeneralRe: c++ Pin
James Curran21-Jun-22 7:29
James Curran21-Jun-22 7:29 
GeneralRe: c++ Pin
tuanguyen200021-Jun-22 14:53
tuanguyen200021-Jun-22 14:53 
GeneralRe: c++ Pin
CPallini21-Jun-22 20:20
mveCPallini21-Jun-22 20:20 
GeneralRe: c++ Pin
tuanguyen200021-Jun-22 22:50
tuanguyen200021-Jun-22 22:50 
GeneralRe: c++ Pin
tuanguyen200021-Jun-22 22:51
tuanguyen200021-Jun-22 22:51 
GeneralRe: c++ Pin
CPallini21-Jun-22 23:08
mveCPallini21-Jun-22 23:08 
GeneralRe: c++ Pin
Richard MacCutchan22-Jun-22 1:20
mveRichard MacCutchan22-Jun-22 1:20 
QuestionMessage Closed Pin
16-Jun-22 5:53
Member 1496877116-Jun-22 5:53 
AnswerRe: parsing interactive text Pin
Mircea Neacsu16-Jun-22 8:09
Mircea Neacsu16-Jun-22 8:09 
AnswerRe: parsing interactive text Pin
trønderen16-Jun-22 10:52
trønderen16-Jun-22 10:52 
Member 14968771 wrote:
is "tokenization" a good search word ??
Fair enough, but that is only the very beginning. A precursor to parsing.

Tokenization is the chopping into atomic pieces of the input text, with no concern for how they are put together. All the tokenizer knows is how to delimit a symbol (token): That a word symbol start with a alphabetic and continues through alphanumerics but ends at the first non-alphanumeric - the tokenizer doesn't know or care whether the word is a variable name, a reserved word or something else. If it finds a digit, it devours digits. If the first non-digit is a math operator or a space, it has found an integer token. If it is a decimal point or an E (and the language permits exponents in literals), the token is a (yet incomplete) float value, and so on. The only language specific thing that the tokenizer needs to know is how to identify the end of a token. Once it has chopped the source code into pieces, its job is done.

Parsing is identifying the structures formed by the tokens. Identifying block, loops, conditional statements etc.

The borderline isn't necessarily razor sharp. Some would say that when the tokenizer finds an integer literal token, it might as well take the the task of converting it to a binary numeric token value, to be handed to the parser. That might be unsuitable in untyped languages where a numeric literal may be treated as a string. After identifying a word symbol, it might search a table of reserved words, possibly delivering it to the parser as a reserved word token. Again, in some languages this is unsuitable (and lots of people would say it goes far beyond a tokenizer's responsibility).

If you want to analyze some input, doing an initial tokenization before starting the actual parsing is a good idea. Most compilers do that.

Curious memory:
One of my fellow students was in his first job after graduation set to identify bacteria in microscope photos. That was done by parsing: They had BNF grammars for different kinds of bacteria, and the image information was parsed according to the various grammars. If the number of parsing errors was too high, the verdict was 'Nope - it surely isn't that kind of bacteria, let me try another one!' Those grammars with a low error count was handed over to a human expert for confirmation, or possibly making a choice between viable alternatives, if two or more grammars gave a low error count. This mechanism took a lot of trivial work off the medical personnel, and the computer could scan far more images for possibly dangerous bacteria than there would be human resources to do. The university lecturer in the Compilers and Compilation course certainly hadn't prepared us for compiling bacteria!
JokeRe: parsing interactive text Pin
Peter_in_278016-Jun-22 11:56
professionalPeter_in_278016-Jun-22 11:56 
AnswerRe: parsing interactive text Pin
Richard MacCutchan18-Jun-22 2:50
mveRichard MacCutchan18-Jun-22 2:50 
QuestionCan importing C++ DLL files in a C# project has a positive effect on the overall performance of the application? Pin
Code4Ever10-Jun-22 8:39
Code4Ever10-Jun-22 8:39 
AnswerRe: Can importing C++ DLL files in a C# project has a positive effect on the overall performance of the application? Pin
Dave Kreskowiak10-Jun-22 9:56
mveDave Kreskowiak10-Jun-22 9:56 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.