PEG Parser (Alpha-2 Version)

I made the peg parser XPEG, it became alpha-2 version now. The parser provides Voice Report, which is my dream! :D

The Voice Report is in the voice.dll as a plug-in of the parser. If you set the -! option to the parser, it reports progress and results by woman voice, which is now Japanese.
I want to provide more DLLs to increase patterns and/or add other voices in future, but now it would be hard ... I am busy to my jobs and I think other analyzing feature is more important than it.

Thanks Chiiko Mitsuhashi. I am glad to use her voices.

TODO

• To make XCP compiler uses XPEG.
• To implement left-recursive detection
• To design Actions interface
• To test more.
• To replace PlaySount to multimedia APIs supporting 16bit PCM (Voice Report is resampled to 8bit PCM now because PlaySound can not play 16bit PCM)

Left Recursive Definition in PEG

PEG parsing progresses from top to down like LL(n) parsing. Thus, its behavior is similar with LL grammars. So, PEG cannot deal with left-recursive definition and it will cause stack overflow at run time.

The stack overflow is serious problem because it occurs at run time and makes target application crash.

To avoid such crash, static diagnosis is effective. So, the PEG parser should check the left-recursive definition on parsing.

As implementation changed from one-pass parsing to two-pass using parsed syntax treee, it will analyze such left-recursive. I would make it...

Hyphen(-) should be put at the Left Bound in a Class of PEG

On PEG, hyphen(-) in a class indicates range of characters. For example, [A-Za-z] means all the alphabets in ASCII characters. Thus, to indicate a character '-', the '-' is put at the bounds of a class, for example, [-A-Za-z]. Then, the '-' should be put at the left bound because the '-' at the right bound is parsed as unexpected.

Regular expression has similar syntax, and some implementations allow both the left and right bounds. So, [-A-Za-z] equals to [A-Za-z-]. (But, I do not know such behavior is valid or not in strict regular expression)

But at least official PEG syntax does not allow the right bound.

Literal <- ['] (!['] Char)* ['] Spacing
/ ["] (!["] Char)* ["] Spacing
Class <- '[' (!']' Range)* ']' Spacing
Range <- Char '-' Char / Char
Char <- '<- ’\\’ [nrt’"\[\]\\]
/ ’\\’ [0-2][0-7][0-7]
/ ’\\’ [0-7][0-7]?
/ !’\\’ .

In the above grammar, although the Class's expression excepts ']' from the class, Range's expression accepts the ']' and '-' as the Char that is following the '-'. Thus, the Char can match the last '-' of [A-Za-z-] and ']' become the second Char of a Range. Then, the class is not enclosed by the ']', which probably causes unexpected behavior on parsing.

On the other hand, as hyphen at the left bound is parsed as not Range but a single Char, such problem does not occur.

As the result, hyphen(-) should be put at the left bound in a class of PEG.

PEG parser

I released XPEG parser version 0.1.2 yesterday. The parser can parse PEG-formated string and generate parser source code in C++ from it. Now, the parser is alpha version and error-handling is weak.

"Programmers' Grimoire" Vol.1

I bought the technical magazine, "Programmers' Grimoire" Vol.1 "Construct the World, C++" , which is written in Japanese and can be downloaded as PDF.

The Vol.1 features C++. I will list its articles here.

1. For First Issue of the "Programmers' Grimoire"
2. Interview to Bjarne Stroustrup
3. Introduction of boost::serialization (First part)
4. Variadic Template - I'd been waiting for it -
5. Thinking aboult type-system using Chrono library
6. "Oven-Range" cooking
7. Hello,C++ World!
8. Crawling in the Stream
9. The height of Meta-programming, Ahh!
10. Introduction of HTTP-communication using Boost.Asio
11. C++'s History
12. Experience Note on BoostCon2010

Windows 7 ...

I bought a new Panasonic mobile PC that preinstalled Windows 7 Professional.

Although Vista did not support VS7.1, the Win7 has XP mode. I tried to install VS7.1 into the XP mode, but warning on known compatibility problem was shown as expected. Although I want to use the VC7.1 as I like VS7.1, the VS7.1 is not supported on XP mode of Win7.　It is regrettable.
Reluctantly, I installed VS10 into the Windows 7.

Because I use not Vista but XP with classic theme for a long time, I get irritated in Win7's UI...

CERT Secure Coding Standards

• CERT C Secure Coding Standards (Japanese ed.)
• CERT C Secure Conding Standard
• CERT C++ Secure Conding Standard
• The CERT Oracle Secure Coding Standard for java

Making PEG Parser

I write my PEG parser to learn the PEG and packrat parsing. Now, the my parser can parse PEG. But now, the parser has not generated source code. I am going to make it generate C++ source code. And now, the parser has not supported packrat parsing. I am implementing memoization.

Memo on PEG Links

Links on PEG

• PEG (Wikipedia)
• The Packrat Parsing and Parsing Expression Grammars Page

Links on PEG (in Japanese)

• Onion wo Kaihatsu-shitsutsu, PEGEX wo Kaihatsu-suru Nikki (Diary on making PEGEX while making Onion)
• PEG no tokucho (PEG's features)
• Kokai-kansu-gata PEG Parser (A PEG Parser using Higher-Order Function)

Fixed Memo on Narwhal #2

Narwhal v.0.2 is released. I updated to the version.

Other found bugs are the following:

The ' " ' rule cannot be compiled as C language

Compiling a parser generated from the grammar that contains rules with double-quote (") fails because generated string literals are not enclosed by " well. For example, the rule ' " ' is generated as L" " ". Furthermore, ' \" ' and " \" " are generated as L" \\" ". I could not escape the double quote.

Escaped character cannot match input.

For example, the '\n' rule, which consists of a single character, cannot match an input single character '\n' in the S macro, because the print_escape function generated the rule as the two characters "\n".
If it is necessary to escape the rule for C language, input characters should be necessary to be escaped or the rule should be restored before matching.
When I tried to escape such characters in input_buffer_read_char, it seems the characters became matched (but, it is not enough tested ).

because my multi-line testing grammar did not work well, I debugged it. But, perhaps because Narwhal's example calculator is single-line, the bugs might be hard to find before. I found the Narwhal's source code is clear and simple C, which is easy to fix. Although I did not provide my fixed code because it contains my customization (it is maybe unnecessary for others), information I think helpful for others was reported to the Narwhal project .

Other local customization

• trace dump is added. It is available if the prefix_debug is not zero.
• Generated time stamp is removed because it is detected as difference.
• other tribal modifications

Fixed Memo on Narwhal

I changed the following:

Customization

• BUF_LEN in c_generator.c was changed to 4096 because stack overflow occurs on generating parser.
• Preprocessor condition on swprintf's parameter was added to generated parser because the warning 4706 occurred and it causes the access violation at runtime.

Fixed Bugs?

• It is necessary to escape double quote (") in print_escape function because it causes such compile errors as C2308, C2001, and C1057 during compiling the generated parser, if grammar includes double quote.
• Generated str_dup and wcs_dup functions became 'static' because compile errors on multiple definition occurred during compiling the generated parser if there are two or more parsers in a program.
• Order of the following expressions in c_generator.c was swapped because the warning 4702 occurred when compiling the generated parser.
  ib->buf_size -= INPUT_BUFFER_SIZE_INCREMENT;\n"
return WEOF;\n"
• Type of 'eol_pos' of find_line_endings generated by print_utility_source function at c_generator.c was fixed to int because the warning 4244 occurred during compiling the generated parser.
• The local variable 'n' of find_line function generated by the print_utility_source function became initialized as -1 because the warning 4701 occurred during compiling the generated parser.

Note

• I set compiler option to suppress the warning 4706 and 4100 during compiling the generated parser.

Narwhal Parser Generator

I am trying other parser, Narwhal.

• The parser is provided under BSD-style license.
• The parser uses not BNF but Parsing Expression Grammer (PEG).
• The parser (and PEG) is not LALR/LL and does not separate lexical analyzer and syntax analyzer.
• The parser uses wchar_t.
• The parser generated C source code.
• The parser builds a syntax tree without user-defined closures and procedures.

Tthe official binary version 0.1 depends on VC8 or later because generated source code uses fopen_s of Secure C Library proposed by Microsoft. But, such dependency was fixed in the version 0.1.2 provided as source code only. Thus, I use the version 0.1.2.
However, there was other issue to use with VC7.1. Although source code generated by the version 0.1.2 can build an execution module, the module caused access violation at parsing. When I debug it, I found that the access violation occurs in swprintf. Its cause was mismatch of function parameters.

Difference of swprinf sigunature between VC8 and VC7.1

In ISO/IEC9899:1999 (C99), the second argument of swprintf is size of the buffer in the first parameter, but in old C library of VC7.1 not supporting the C99, such size parameter did not exist, which causes the access violation.
Furthermore, in not C but C++ of VC7.1, the second parameter is supported because standard C++ refers to the version C99 explicitly and requires the compatible library with C99. So, implementation of VC7.1 uses #ifdef __cplusplus. As, the parser generates source code in not C++ but C, the access violation occurred.
On the other hand, in VC9, there is not such problem because the second parameter of swprintf has been supported.

BOOST_SPIRIT_LEXERTL_DEBUG Is For Only Char

Code using wchar_t does not work well.

#if defined(BOOST_SPIRIT_LEXERTL_DEBUG)
                boost::lexer::debug::dump(state_machine_, std::cerr);
#endif

develog - C++: Boost.Spirit in_state with Wide Character

develog - C++: Boost.Spirit in_state with Wide Character

I do try-and-errors with test code based the example3. But, I cannot succeed to compile it. I tried the other example lex::example5, but it is failed, too.

If you are seeing a compilation error here stating that the forth parameter can't be converted to a qi::reference then you are probably trying to use a rule or a grammar with an incompatible skipper type.

I read the comment in the source, which is my problem. But, I don't know how it do such.

2010/12/06

The following change works well using boost.1.45.0
in_state_skipper<Lexer, wchar_t>

Boost.Spirit in_state with Wide Character

I am writing grammar in wide characters. The following code is example3 of Spirit using wchar_t instead of char. (example.hpp is expended here.) Now this in_state seems not to work well using wchar_t.

The in_state causes compiling errors by the code.

Sure, in_state_skipper template class requires the template parameters, template <typename Skipper, typename String = char const*>, which String seems necessary to wide character instead of the single byte character. But, may the Skipper is something right?

//  Copyright (c) 2001-2010 Hartmut Kaiser
// 
//  Distributed under the Boost Software License, Version 1.0. (See accompanying 
//  file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

//  This example shows how to create a simple lexer recognizing a couple of 
//  different tokens and how to use this with a grammar. This example has a 
//  heavily backtracking grammar which makes it a candidate for lexer based 
//  parsing (all tokens are scanned and generated only once, even if 
//  backtracking is required) which speeds up the overall parsing process 
//  considerably, out-weighting the overhead needed for setting up the lexer.
//
//  Additionally, this example demonstrates, how to define a token set usable 
//  as the skip parser during parsing, allowing to define several tokens to be 
//  ignored.
//
//  This example recognizes couplets, which are sequences of numbers enclosed 
//  in matching pairs of parenthesis. See the comments below to for details
//  and examples.

// #define BOOST_SPIRIT_LEXERTL_DEBUG
// #define BOOST_SPIRIT_DEBUG

#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/lex_lexertl.hpp>

#include <iostream>
#include <fstream>
#include <string>

//#include "example.hpp"
//  Copyright (c) 2001-2010 Hartmut Kaiser
//  Copyright (c) 2001-2007 Joel de Guzman
// 
//  Distributed under the Boost Software License, Version 1.0. (See accompanying 
//  file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

#include <iostream>
#include <fstream>
#include <string>

#if 1
typedef wchar_t  Ch;
#define C(s)  L##s
#define Err   wcerr
#else
typedef char  Ch;
#define C(s)  s
#define Err   cerr
#endif

///////////////////////////////////////////////////////////////////////////////
//  Helper function reading a file into a string
///////////////////////////////////////////////////////////////////////////////
inline std::basic_string<Ch>
read_from_file(char const* infile)
{
    std::basic_ifstream<Ch> instream(infile);
    if (!instream.is_open()) {
        std::Err << C("Couldn't open file: ") << infile << std::endl;
        exit(-1);
    }
    instream.unsetf(std::ios::skipws);      // No white space skipping!
    return std::basic_string<Ch>(std::istreambuf_iterator<Ch>(instream.rdbuf()),
                       std::istreambuf_iterator<Ch>());
}



using namespace boost::spirit;

///////////////////////////////////////////////////////////////////////////////
//  Token definition
///////////////////////////////////////////////////////////////////////////////
template <typename Lexer>
struct example3_tokens : lex::lexer<Lexer>
{
    example3_tokens()
    {
        // define the tokens to match
        ellipses = C("\\.\\.\\.");
        number = C("[0-9]+");

        // associate the tokens and the token set with the lexer
        this->self = ellipses | C('(') | C(')') | number;

        // define the whitespace to ignore (spaces, tabs, newlines and C-style 
        // comments)
        this->self(C("WS")) 
   =   lex::token_def<lex::unused_type, Ch>(C("[ \\t\\n]+"))          // whitespace
            |   C("\\/\\*[^*]*\\*+([^/*][^*]*\\*+)*\\/")   // C style comments
            ;
    }

    // these tokens expose the iterator_range of the matched input sequence
 lex::token_def<lex::unused_type, Ch> ellipses, identifier, number;
};

///////////////////////////////////////////////////////////////////////////////
//  Grammar definition
///////////////////////////////////////////////////////////////////////////////
template <typename Iterator, typename Lexer>
struct example3_grammar 
  : qi::grammar<Iterator, qi::in_state_skipper<Lexer> >
{
    template <typename TokenDef>
    example3_grammar(TokenDef const& tok)
      : example3_grammar::base_type(start)
    {
        start
            =  +(couplet | tok.ellipses)
            ;

        //  A couplet matches nested left and right parenthesis.
        //  For example:
        //    (1) (1 2) (1 2 3) ...
        //    ((1)) ((1 2)(3 4)) (((1) (2 3) (1 2 (3) 4))) ...
        //    (((1))) ...
        couplet
            =   tok.number
            |   C('(') >> +couplet >> C(')')
            ;

        BOOST_SPIRIT_DEBUG_NODE(start);
        BOOST_SPIRIT_DEBUG_NODE(couplet);
    }

    qi::rule<Iterator, qi::in_state_skipper<Lexer> > start, couplet;
};

///////////////////////////////////////////////////////////////////////////////
int main()
{
    // iterator type used to expose the underlying input stream
    typedef std::basic_string<Ch>::iterator base_iterator_type;

    // This is the token type to return from the lexer iterator
    typedef lex::lexertl::token<base_iterator_type> token_type;

    // This is the lexer type to use to tokenize the input.
    // Here we use the lexertl based lexer engine.
    typedef lex::lexertl::lexer<token_type> lexer_type;

    // This is the token definition type (derived from the given lexer type).
    typedef example3_tokens<lexer_type> example3_tokens;

    // this is the iterator type exposed by the lexer 
    typedef example3_tokens::iterator_type iterator_type;

    // this is the type of the grammar to parse
    typedef example3_grammar<iterator_type, example3_tokens::lexer_def> example3_grammar;

    // now we use the types defined above to create the lexer and grammar
    // object instances needed to invoke the parsing process
    example3_tokens tokens;                         // Our lexer
    example3_grammar calc(tokens);                  // Our parser

    std::basic_string<Ch> str (read_from_file("example3.input"));

    // At this point we generate the iterator pair used to expose the
    // tokenized input stream.
    std::basic_string<Ch>::iterator it = str.begin();
    iterator_type iter = tokens.begin(it, str.end());
    iterator_type end = tokens.end();

 std::basic_string<Ch> ws = C("WS");

    // Parsing is done based on the the token stream, not the character 
    // stream read from the input.
    // Note how we use the lexer defined above as the skip parser.
 bool r = qi::phrase_parse(iter, end, calc,
//  qi::in_state< Ch const*>( C("WS"))[tokens.self] );
//  qi::in_state( C("WS"))[tokens.self] );
  qi::in_state(ws)[tokens.self] );

    if (r && iter == end)
    {
        std::cout << "-------------------------\n";
        std::cout << "Parsing succeeded\n";
        std::cout << "-------------------------\n";
    }
    else
    {
        std::cout << "-------------------------\n";
        std::cout << "Parsing failed\n";
        std::cout << "-------------------------\n";
    }

    std::cout << "Bye... :-) \n\n";
    return 0;
}

VC9 tell me the following:

error C2440 at lex/qi/state_switcher.hpp

'conversion' : cannot convert from 'const wchar_t *' to 'const char *const'.

at /qi/nonterminal/rule.hpp

'boost::spirit::qi::state_switcher_context<Subject,State>::state_switcher_context<const std::basic_string<_Elem,_Traits,_Ax>>(const boost::spirit::qi::state_switcher_context<Subject,const std::basic_string<_Elem,_Traits,_Ax>> &)' with Subject=boost::spirit::lex::reference<const boost::spirit::lex::detail::lexer_def_<boost::spirit::lex::lexer>>, State=const char *const , Elem=wchar_t, Traits=std::char_traits<wchar_t>, _Ax=std::allocator<wchar_t> ]

at qi/reference.hpp

'bool boost::spirit::qi::rule<Iterator,T1,T2,T3>::parse<Context,Skipper,Attribute>(Iterator &,const Iterator &,Context &,const Skipper &,Attribute &) const'

....

error C2439 at lex/qi/state_switcher.hpp

member could not be initialized
'boost::spirit::qi::state_switcher_context<Subject,State>::state' :
with
[
Subject=boost::spirit::lex::referencelt;const boost::spirit::lex::detail::lexer_def_lt;boost::spirit::lex::lexer>>, State=const char *const

at lex/qi/state_switcher.hpp

'boost::spirit::qi::state_switcher_contextlt;Subject,State>::state'
with
Subject=boost::spirit::lex::referencelt;const boost::spirit::lex::detail::lexer_def_lt;boost::spirit::lex::lexerlt;lexer_type>>>, State=const char *const

state_switcher_context's State is char...

qi::in_state<example3_tokens::lexer_def, Ch>(ws)[tokens.self] );

• error C2275: 'example3_tokens' : illegal use of this type as an expression
• error C2679: binary '[' : binary 'operator' : no operator defined which takes a right-hand operand of type 'boost::spirit::lex::detail::lexer_def_<LexerDef>' (or there is no acceptable conversion)

qi::in_state<example3_tokens::lexer_def(), Ch>(ws)[tokens.self] );

• error C2512: 'boost::spirit::lex::detail::lexer_def_<LexerDef>::lexer_def_' : no appropriate default constructor available
  with
  [
  LexerDef=boost::spirit::lex::lexer
  ]
• error C2679: ...

Explicit template parameter would not be used. I tried this on compiled single byte character version:
std::Err << typeid(qi::in_state(ws)).name() << std::endl << typeid(qi::in_state(ws)[tokens.self]).name() << std::endl;
The result is this:

struct boost::proto::exprns_::expr<struct boost::proto::tag::terminal,struct boo st::proto::argsns_::term<struct boost::spirit::terminal_ex<struct boost::spirit: :tag::in_state,struct boost::fusion::vector1<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > > > >,0> struct boost::proto::exprns_::expr<struct boost::proto::tag::subscript,struct bo ost::proto::argsns_::list2<struct boost::proto::exprns_::expr<struct boost::prot o::tag::terminal,struct boost::proto::argsns_::term,class std::allocator<char> > > > >,0> &,struct boost::spirit::lex::detail::lexer_def_<class boost::spirit::le x::lexer,class std::allocator<char> >,struct boost::mpl::vector0<struct boost::mpl::na>,struct boost::mpl::bool_<1> >,class std::_String_iterator<char,struct std::char_traits <char>,class std::allocator<char> >,class boost::spirit::lex::lexertl::functor<s truct boost::spirit::lex::lexertl::token<class std::_String_iterator<char,struct std::char_traits<char>,class std::allocator<char> >,struct boost::mpl::vector0< struct boost::mpl::na>,struct boost::mpl::bool_<1> >,struct boost::spirit::lex:: lexertl::detail::data,class std::_String_iterator<char,struct std::char_traits<c har>,class std::allocator<char> >,struct boost::mpl::bool_<0>,struct boost::mpl: :bool_<1> > > > > &>,2>

To Use Spilit with Wide Character

Common

• Use wchar_t instead of char.
• Set your locale if default C locale is not enough to express your language.
• Use std::wstring instead of std::string.
• Use std::wiostream family instead of std::iostream

Of course, you can use std::basic_string<...> and std::basic_stream<...> family directly, or use typedef.

Lex

• Use boost::spirit::lex::token_def<std::wstring, wchar_t> instead of token_def<>.
• Write regular expressions in wide characters.
• Use boost::spirit::lex::lexertl::token<wchar_t const*> instead of token<char const*>.

Qi

• Use boost::spirit::qi::symbols<wchar_t,std::wstring instead of symbols<char,std::string>
• Write grammar rules in wide characters.
• Use std::wstring::iterator instead of std::string::iterator
• As skipper also requires wide character version, replace skipper parameter of phrase_parse().

Note

• As the argument of boost::spirit::qi::rule<...>::name() is not depends on character set, it is only single byte characters.

The Battle for Wesnoth

I found the fantasy tactical simulation game, the Battle for Wesnoth. As I like tactical simulation games with turn-phase system, I played it. Although its system is simple, it is good. I feel that its developers know analog war games with board, chip, and dice well.

• Simulated results are shown before attack.
• Internationalized. I can select English or Japanese (I cannot read other languages).
• There are Zone of Control (ZOC), sight, Moving points in a turn.
• It is online game with GPL. Although I have not read its source code, I think that non-realtime system can be harder to cheat than realtime by design.
• I have seen save files archived ".gz" of game at first. I found that GPL allows reverse engineering including gaming data.

Fatal Error C1001 of VC 2008

I am fixing Fatal Error C1060. But now, Fatal Error C1001 (Internal Compile Error) occurs... hummm. Although there might be bugs in the code, for example recursive reference in the grammar, compiler's fatal error does not tell me right reason and provide useful information. I have to check it more...

Fatal Error C1060 of VC 2008

The fatal error C1060 occurs when I write certain grammar.

/boost/type_traits/remove_const.hpp

BOOST_TT_AUX_TYPE_TRAIT_DEF1(remove_const,T,typename boost::detail::remove_const_impl<T>::type)

C1060 compiler is out of heap space

The operating system or run-time library cannot fill a request for memory.

Possible solutions

• Use /Zm to increase memory allocation limit.
• Increase the size of the Windows swap-file.
• Close other running programs.
• Eliminate unnecessary include files.
• Eliminate unnecessary global variables.(For example, by allocating memory dynamically instead of declaring a large array.) Eliminate unused declarations.
• Split the current file into smaller files.

Although the error has occurred before with some versions of Visual Studio (VC), the /Zm800 option could resolve it. But in this case, the option cannot resolve this error. The cl.exe uses about 1.6GB memory when I see by Task Manager. Furthermore, VC9's cl.exe sometimes causes application error during compiling this code, which is perhaps too large. Compiling XP has 3GB memory and other huge programs are not running. I will have to split the code.

I have not written separated rules in Spirit 2.x and not use Proto directly. I will study it.

The Memo of a Mistake on std::functional

Although I usually write functors by boost::bind or boost::lambda::bind, I wrote these by standard templates in the last weekend. At that time, I had following easy mistake.

std::vector<zzz> xxx;
std::find(xxx.begin(), xxx.end(), std::bind2nd(&zzz.foo, yyy);

I spent some time to find the reason of this compilation error.

If I would write it with boost, I write it like the following:

namespace bl = boost::lambda;
bl::bind(&zzz.foo, bl::_1, yyy);,
which is the reason of this mistake.

So, I forgot that it is necessary to std::mem_fun.

std::find(xxx.begin(), xxx.end(), std::bind2nd(std::mem_fun(&zzz.foo), yyy);

Programming is often restricted on using third-party's library by something; For example, portability, license, and/or contract. Although for example VC9 has std::tr1 including std::tr1::bind, I am sorry it cannot always be used. In such cases, non-restricted Standard library is very important, which is more troublesome than the Boost just a little now.

Link list to Spirit's References

Qi

• Parser Directives
• Compound Attribute Rules
• Parser Operators
• Numeric parsers
• Character parsers
• Semantic actions

Lex

• Supported Regular Expressions

Spirits-related Links in Japanese

Shift-JIS parser for Spirit 2.1

I found out the site, Mui-Kukan, which shows Shift-JIS parser example for Spirit.2.1.

Let's boost

Let's boost is a famous site on Boost in Japanese. I have a book Mr. Inaba wrote, the owner of the site. In this page, the version of Spirit is 2.x.

C++ Labyrinth

C++ Labyrinth is also famous site on C++. Spirit in this page is 1.x.

fork-exec problem

memologue (<- good article but in Japanese)

1. Thread-1 and thread-2 are running in parent process before forking.
2. The thread-1 calls doit function.
3. The doit function locks own mutex.
4. The thread-1 sleeps by nanosleep.
5. Then, the current context switches to thread-2.
6. Thread-2 calls fork function.
7. A child process is created.
8. In this time, doit's mutex in the child process has been locked (because it was copied from locking parent's); however, there is not any thread to unlock the mutex in the child process!
9. The child process starts processing.
10. In the child process, doit function is called.
11. The doit attempts to lock the mutex already locked, which causes dead-lock in the child process.

Solutions

1. Exit all other threads before forking.
2. Use exec after forking soon in new child processes.
4. Use only fork-safe procedures in all the other threads.
5. Use callback functions before and after forking with pthread_atfork.
6. No forking in multi-thread program.

Spirit 2.1 of Boost 1.41 and VC7.1

Spirit 2.1 of Boost 1.41 did not work well with VC71.
Although I fixed some problems on difference of the version of Spirit at this weekend, other unknown internal errors of VC71 occurred and I could not fix them. I think these are probably problems or limitations on VC71 because the same source code has no problem using VC9.
Hence I gave up compiling Spirit 2.1 with VC71 and decided to use VC9 instead of VC71 for Spirit 2.1 .

Sure, VC71 was released in 2003 and has become too old, so I am waiting for the release of VC2010 nowadays. Since I almost all use pure standard C++ only, I look forward to such features of VC2010.

Fatal error occurs in exception of Boost 1.41

I changed Boost from 1.40 to 1.41. Then, the Boost causes fatal error at exception_ptr.hpp: 160:

160: #ifndef BOOST_NO_RTTI
161: (*this) << original_exception_type(&typeid(e));
162: #endif


... may VC7.1 be cause of it?

I must define BOOST_NO_RTTI although RTTI option of VC (GR) is available.

Boost 1.41 Update

I updated version of Boost to 1.41. Then, Boost.Spirit is updated to 2.1 and I am applying code of mine to it.

Boost.asio.ip.tcp.iostream

1. sending to a stream is blocking during receiving from the stream even on other thread.

2. duplication of stream socket failed.
iostream is;
iostream os;
os.rdbuf().assign(is.rdbuf().native());
Then, the assignment failed because the socket has been registered.