What is gcc really saying here?

If you look at line 6633 in toke.c, you'll see this line of code:

if (memchr(tmpbuf, ':', len))

Looks innocent enough, and I don't see any '&' signs in there at all! I spent some time mucking with different compiler settings, until I found that the same C file compiled just fine on my MacBook, which is 32-bit, and configured to produce a 32-bit perl. Since the files were the same on both machines, I knew that I had some sort of #define issue, caused by a header file.

However, since I didn't know which header file or #define could be causing the issue, I looked at the C-preprocessor version of the file (generated by passing the '-E' flag to gcc) in order to see the actual line of code that was failing. Because of all of the preprocessor branches in toke.c, the pre-processed version of the file looked quite different that the raw source file. It was also over 20,000 lines long. In order to find the offending line of code, I actually needed to have three different versions of the file open - the preprocessor output from my 32-bit build (which worked), from the 64-bit build (didn't work), and the original source file. My editor of choice doesn't parse the C code as you edit it, so it can't tell you what function you're in. The 'Perl_yylex()' is crazy-long, so I needed an editor which could tell me which function my cursor is currently in. Hence, I opened up the preprocessed files in Xcode.

With all of this in place, I was able to determine that the above line of code got translated to the following in my 64-bit build:

if (Perl_ninstr((char*)(tmpbuf), ((char*)(tmpbuf)) + len, &(':'), &(':') + 1))

While I'm not a C guru by any stretch, I think I can see here what gcc is complaining about - that &(':') stuff looks totally whack. But, this line of code gave me something to grep for - looking for Perl_ninstr, I found:

amy:$ grep Perl_ninstr *.h
embed.h:#define ninstr			Perl_ninstr
embed.h:#define ninstr(a,b,c,d)		Perl_ninstr(aTHX_ a,b,c,d)
proto.h:PERL_CALLCONV char*	Perl_ninstr(pTHX_ const char* big, const char* bigend, ...

The first hit is interesting - basically perl is defining ninstr() to point to Perl_ninstr(). Looking for just the string "ninstr" reveals:

amy:$ grep ninstr *.h
embed.h:#define ninstr			Perl_ninstr
embed.h:#define rninstr			Perl_rninstr
embed.h:#define ninstr(a,b,c,d)		Perl_ninstr(aTHX_ a,b,c,d)
embed.h:#define rninstr(a,b,c,d)	Perl_rninstr(aTHX_ a,b,c,d)
perl.h:#       define memchr(s,c,n) ninstr((char*)(s), ((char*)(s)) + n, &(c), &(c) + 1)
proto.h:PERL_CALLCONV char*	Perl_ninstr(pTHX_ const char* big, const char* bigend, ...
proto.h:PERL_CALLCONV char*	Perl_rninstr(pTHX_ const char* big, const char* bigend, ...

The hit from 'perl.h' is interesting. Here is the full snippet of code:

#ifndef PERL_MICRO
#ifndef memchr
#   ifndef HAS_MEMCHR
#       define memchr(s,c,n) ninstr((char*)(s), ((char*)(s)) + n, &(c), &(c) + 1)
#   endif
#endif
#endif

Bingo! This code in 'perl.h' is what is making the memchr() function go all unbuildy on my 64-bit machine. But, this will only happen if the "HAS_MEMCHR" preprocessor variable isn't defined. Sure enough, on the 32-bit machine:

areitz@rosetta:/tmp/perl-5.10.0$ grep HAS_MEMCHR config.h
/* HAS_MEMCHR:
#define HAS_MEMCHR	/**/

And on the 64-bit machine:

>amy:$ grep HAS_MEMCHR config.h /* HAS_MEMCHR: /*#define HAS_MEMCHR / **/

Drat! Well, I know that Mac OS X has the 'memchr()' function, even in 64-bit mode. But for some reason, perl thinks that it doesn't, and is trying to substitute in some whack code to compensate. I tried just uncommenting this, and managed to get 'toke.c' to compile. However, a few files later, I encountered this error:

`sh  cflags "optimize='-O3'" util.o`  util.c
	  CCCMD =  cc -DPERL_CORE -c -fno-common -DPERL_DARWIN -no-cpp-precomp \
         -arch ppc64 -fno-common -DPERL_DARWIN -no-cpp-precomp -arch x86_64 \
         -I/usr/local/include  -O3  -Wall 
util.c:1854: error: conflicting types for 'vsprintf'
util.c: In function 'vsprintf':
util.c:1855: error: storage size of 'fakebuf' isn't known
util.c:1869: error: '_IOWRT' undeclared (first use in this function)
util.c:1869: error: (Each undeclared identifier is reported only once
util.c:1869: error: for each function it appears in.)
util.c:1870: warning: implicit declaration of function '_doprnt'
util.c:1855: warning: unused variable 'fakebuf'
util.c:1854: error: conflicting types for 'vsprintf'

Examining this code in 'util.c', I found this helpful comment block:

/* This vsprintf replacement should generally never get used, since
   vsprintf was available in both System V and BSD 2.11.  (There may
   be some cross-compilation or embedded set-ups where it is needed,
   however.)

   If you encounter a problem in this function, it's probably a symptom

   that Configure failed to detect your system's vprintf() function.

   See the section on "item vsprintf" in the INSTALL file.

   This version may compile on systems with BSD-ish ,

   but probably won't on others.

*/

This is where I found what I mentioned above - that the 'Configure' script was having trouble detecting libc functions. I thought about just commenting in the right functions in the 'config.h' file - but I quickly found that perl thought that there were a lot of missing functions - well over 200 by my count. Since I didn't want to uncomment over 200 lines of C, I decided to muck around with the 'Configure' script until I found the right options, which I have highlighted above.