Problems with fastcgi php migration

On Thu, Mar 13, 2008 at 10:37:56AM -0400, Ian M. Evans wrote:

   resulted in:

/fest/tribeca/2003/2/photos <-- working
/fest/tribeca/2003/2/photos/ <-- No input file specified.
/fest/tribeca/2003/2/photos/1 <-- No input file specified.

Would you like to see the debug?

No, debug does not help here.

On Thu, Mar 13, 2008 at 10:03:32AM -0400, Ian M. Evans wrote:

10085#0: *741 generic phase: 2
10085#0: *741 http script complex value
10085#0: *741 http script capture: “photos”
10085#0: *741 http script set var
10085#0: *741 http script complex value
10085#0: *741 http script capture: “/1”

The rewrite module debug log is as ugly as it is itself.
Here I see that

  1. $script_name get “photos” capture, and it probaly equal to “/photos”.
  2. $path_info if “/1”.

10085#0: *741 http script copy: “”
10085#0: *741 http script copy: “”
10085#0: *741 http script var: “”
10085#0: *741 socket 16
10085#0: *741 chain writer in: 0825F490
10085#0: *741 recv: fd:16 128 of 4096
10085#0: *741 http fastcgi header: “Status: 404 Not Found”
And PHP sends 404.
Probably it tries to work with

fastcgi_param PATH_TRANSLATED $document_root$fastcgi_script_name;

or

fastcgi_param REQUEST_URI $request_uri;

Try to delete them and use

fastcgi_param SCRIPT_FILENAME $document_root$script_name;
fastcgi_param PATH_INFO $path_info;

only.

I’ve been poring over the debug logs after trying test cases. I don’t
know if I found anything interesting here, but thought I’d pass it on.

Here’s one that worked, the /galleries in root:

24254#0: *49095 http request line: “GET /galleries/29/1/525 HTTP/1.1”
24254#0: *49095 http uri: “/galleries/29/1/525”
24254#0: *49095 http args: “”
24254#0: *49095 http exten: “”
24254#0: *49095 event timer del: 15: 2848432589
24254#0: *49095 generic phase: 0
24254#0: *49095 find location for “/galleries/29/1/525”
24254#0: *49095 find location: “/”
24254#0: *49095 find location: ~ “.(shtml|php)$”
24254#0: *49095 find location: ~
“^/(testgalleries|galleries|poll|news|photos)(/|$)”
24254#0: *49095 using configuration
“^/(testgalleries|galleries|poll|news|photos)(/|$)”
24254#0: *49095 http cl:-1 max:1048576
24254#0: *49095 generic phase: 2
24254#0: *49095 http script complex value
24254#0: *49095 http script var: “/galleries/29/1/525”
24254#0: *49095 http script set var
24254#0: *49095 http script value: “”
24254#0: *49095 http script set var
24254#0: *49095 http script var
24254#0: *49095 http script var: “/galleries/29/1/525”
24254#0: 49095 http script regex: "^(/[^/]+)(/.)"
24254#0: *49095 http script if
24254#0: *49095 http script complex value
24254#0: *49095 http script capture: “/galleries”
24254#0: *49095 http script set var
24254#0: *49095 http script complex value
24254#0: *49095 http script capture: “/29/1/525”

And here’s one that failed, the php file photos two down in a subdir.
The one major difference: It has a http script copy: "/ MSIE " before
the script capture that the working debug didn’t have…is that
anything?

24255#0: *49353 http request line: “GET /academy/80/photos/106 HTTP/1.1”
24255#0: *49353 http uri: “/academy/80/photos/106”
24255#0: *49353 http args: “”
24255#0: *49353 http exten: “”
24255#0: *49353 event timer del: 12: 2848524365
24255#0: *49353 generic phase: 0
24255#0: *49353 find location for “/academy/80/photos/106”
24255#0: *49353 find location: “/”
24255#0: *49353 find location: ~ “.(shtml|php)$”
24255#0: *49353 find location: ~
“^/(testgalleries|galleries|poll|news|photos)(/|$)”
24255#0: *49353 find location: ~
“/(testgalleries|galleries|poll|news|photos|test)(/|$)”
24255#0: *49353 using configuration
“/(testgalleries|galleries|poll|news|photos|test)(/|$)”
24255#0: *49353 http cl:-1 max:1048576
24255#0: *49353 generic phase: 2
24255#0: *49353 http script complex value
24255#0: *49353 http script var: “/academy/80/photos/106”
24255#0: *49353 http script set var
24255#0: *49353 http script value: “”
24255#0: *49353 http script set var
24255#0: *49353 http script var
24255#0: *49353 http script var: “/academy/80/photos/106”
24255#0: 49353 http script regex:
"^/(?:.+/)?(testgalleries|galleries|poll|news|photos|test)(/.
)"
24255#0: *49353 http script if
24255#0: *49353 http script complex value
24255#0: *49353 http script copy: "/ MSIE "

^-- The line above doesn’t appear in the working debug.

24255#0: *49353 http script capture: “/photos”
24255#0: *49353 http script set var
24255#0: *49353 http script complex value
24255#0: *49353 http script capture: “/106”

Igor,

Hadn’t noticed that in the:

if ($uri ~ ^/(?:.+/)?(testgalleries|galleries|poll|news|photos)(/.*)) {

you had changed:

set $script_name $1; to set $script_name /$1;

Made that change…still no go.

Added “test” to the file list above. Tossed nothing into it except for

<?php phpinfo(); ?>

Going to /subdir/subdir/test showed the phpinfo.

/subdir/subdir/test/ and /subdir/subdir/test/1 both showed nothing.

In all three cases though, the access log showed 200s, no 404s.

I’m staring at all the regexes and can’t see how we’ve got
/galleries, /galleries/, and /galleries/24/1/1 all working fine but this
new regex is working on /subdir/subdir/photos but tripping up on
/subdir/subdir/photos/ and /subdir/subdir/photos/1

Again, thanks for nginx and all the help. I’m moving fast on this
because one of the ad networks will be featuring us soon and I’d love to
be 100% nginx by then.

Just wanted to reiterate my activities tonight.

I’ve spent so much time looking at the debug logs this evening, I feel
like I’ve become one with my monitor.

In the location that Igor suggested for finding an extensionless php
file in multiple sub dirs, nginx has been able to find and run the file
at:
/subdir/subdir/test

A-OK, file runs as expected, people cheer in the streets.

I changed the test file from it’s original script to just a phpinfo();
to make sure nothing in the script could have been causing the failure.

/subdir/subdir/test gave us the phpinfo()

/subdir/subdir/test/ and /subdir/subdir/test/1 create a blank screen.
There’s a 200 OK response, the fastcgi server sends:
http fastcgi header: “X-Powered-By: PHP/5.2.5”
http fastcgi parser: 0
http fastcgi header: “Content-type: text/html” and the static location
even sends the favicon.ico over. Just no phpinfo() or any html of any
kind.

I’m perplexed.

I’ve also noticed weird things in the script capture portion of the
debug.

On /subdir/subdir/test/ <–with trailing slash, there was:
32720#0: 75364 http script regex:
"^/(?:.+/)?(testgalleries|galleries|poll|news|photos|test)(/.
)"
32720#0: *75364 http script if
32720#0: *75364 http script complex value
32720#0: *75364 http script copy: “/hars” <-- what’s this?
32720#0: *75364 http script capture: “/test”
32720#0: *75364 http script set var
32720#0: *75364 http script complex value
32720#0: *75364 http script capture: “/”
32720#0: *75364 http script set var

Where did /hars come from?

On /subdir/subdir/test/1 there was:

32720#0: 76311 http script regex:
"^/(?:.+/)?(testgalleries|galleries|poll|news|photos|test)(/.
)"
32720#0: *76311 http script if
32720#0: *76311 http script complex value
32720#0: *76311 http script copy: “/” sr" <-- what’s this?
32720#0: *76311 http script capture: “/test”
32720#0: *76311 http script set var
32720#0: *76311 http script complex value
32720#0: *76311 http script capture: “/1”

Where did “/” sr" come from. I’m guessing this weird extra info may be
screwing things up?

Thanks to all on the list for putting up with this long thread. I’ve
been running the extensionless php files (in any subdir) for almost a
decade with the simple Apache directive:

ForceType application/x-httpd-php

I didn’t realize that getting the same thing to work in nginx would be
such a mindbender, but I’m willing to stare at the monitor and try all
of Igor’s and the list’s suggestions because the speed, memory footprint
and general beauty of nginx make the journey worthwhile. Please keep
those suggestions coming. :slight_smile:

As I wrote to Igor earlier today, we’re 99.99% there and I’m sure we’ll
make it. I know I’m not the only one in the Apache world that’s used
extensionless php scripts so I’m sure this will finally make a wiki
entry that will encourage more folks to make the jump to nginx.

As a reminder, here’s Igor’s latest suggestion for the location:

location ~ /(testgalleries|galleries|poll|news|photos|test)(/|$) {
root /usr/local/apache/htdocs;
set $script_name $uri;
set $path_info “”;

if ($uri ~
^/(?:.+/)?(testgalleries|galleries|poll|news|photos|test)(/.*)) {
set $script_name /$1;
set $path_info $2;
fastcgi_pass 127.0.0.1:10004;
}

fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
#fastcgi_param PATH_TRANSLATED $document_root$fastcgi_script_name;
#fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
fastcgi_param SCRIPT_FILENAME $document_root$script_name;
fastcgi_param PATH_INFO $path_info;
fastcgi_param REDIRECT_STATUS 200;
fastcgi_pass 127.0.0.1:10004;
}

I wondered briefly if:
set $script_name /$1; was a typo but changing it to $1 doesn’t fix it
either.

Thanks everyone.

Hello!

On Sat, Mar 15, 2008 at 11:56:25AM -0400, Ian M. Evans wrote:

30680#0: *1174 fastcgi param: “SERVER_SOFTWARE: nginx”

Great. Scriptname is correct. Root’s correct. Path_info is empty. It works.

Now I called /academy/80/test/12

phpinfo() should come up and the only major change is that path_info should
be 12.

The debug logs show that it’s passing a path_info of /12
The script filename is correct. But phpinfo() doesn’t run.

[…]

30680#0: *1202 fastcgi param: “SCRIPT_FILENAME:
/usr/local/apache/htdocs/test”

Looks like you’ve got wrong SCRIPT_FILENAME. At least - this
differs from the one in the above request.

Maxim D.

Banging my head on the desk here. I added ‘test’ to the list of
filenames that can be extensionless. Placed it a few subdirs down.

All ‘test’ contains is the phpinfo();

When I go to /academy/80/test, the phpinfo() comes up. Here are the
params from the nginx debug:

Parameters from working page:
30680#0: *1174 fastcgi param: “GATEWAY_INTERFACE: CGI/1.1”
30680#0: *1174 fastcgi param: “SERVER_SOFTWARE: nginx”
30680#0: *1174 fastcgi param: "QUERY_STRING: "
30680#0: *1174 fastcgi param: “REQUEST_METHOD: GET”
30680#0: *1174 fastcgi param: "CONTENT_TYPE: "
30680#0: *1174 fastcgi param: "CONTENT_LENGTH: "
30680#0: *1174 fastcgi param: “DOCUMENT_URI: /academy/80/test”
30680#0: *1174 fastcgi param: “DOCUMENT_ROOT: /usr/local/apache/htdocs”
30680#0: *1174 fastcgi param: “SCRIPT_FILENAME:
/usr/local/apache/htdocs/academy/80/test”
30680#0: *1174 fastcgi param: "PATH_INFO: "
30680#0: *1174 fastcgi param: “REDIRECT_STATUS: 200”

Great. Scriptname is correct. Root’s correct. Path_info is empty. It
works.

Now I called /academy/80/test/12

phpinfo() should come up and the only major change is that path_info
should be 12.

The debug logs show that it’s passing a path_info of /12
The script filename is correct. But phpinfo() doesn’t run.

30680#0: *1202 fastcgi param: “GATEWAY_INTERFACE: CGI/1.1”
30680#0: *1202 http script copy: “”
30680#0: *1202 fastcgi param: “SERVER_SOFTWARE: nginx”
30680#0: *1202 http script copy: “”
30680#0: *1202 fastcgi param: "QUERY_STRING: "
30680#0: *1202 http script copy: “”
30680#0: *1202 http script var: “”
30680#0: *1202 fastcgi param: “REQUEST_METHOD: GET”
30680#0: *1202 http script copy: “”
30680#0: *1202 fastcgi param: "CONTENT_TYPE: "
30680#0: *1202 http script copy: “”
30680#0: *1202 fastcgi param: "CONTENT_LENGTH: "
30680#0: *1202 http script copy: “”
30680#0: *1202 http script var: “”
30680#0: *1202 fastcgi param: “DOCUMENT_URI: /academy/80/test/12”
30680#0: *1202 http script copy: “”
30680#0: *1202 http script var: “”
30680#0: *1202 fastcgi param: “DOCUMENT_ROOT: /usr/local/apache/htdocs”
30680#0: *1202 http script copy: “”
30680#0: *1202 http script var: “”
30680#0: *1202 fastcgi param: “SERVER_PROTOCOL: HTTP/1.1”
30680#0: *1202 http script copy: “”
30680#0: *1202 http script var: “”
30680#0: *1202 fastcgi param: “REMOTE_PORT: 2487”
30680#0: *1202 http script copy: “”
30680#0: *1202 http script var: “”
30680#0: *1202 fastcgi param: “SERVER_PORT: 8088”
30680#0: *1202 http script copy: “”
30680#0: *1202 http script var: “”
30680#0: *1202 fastcgi param: “SCRIPT_FILENAME:
/usr/local/apache/htdocs/test”
30680#0: *1202 http script copy: “”
30680#0: *1202 http script var: “”
30680#0: *1202 fastcgi param: “PATH_INFO: /12”
30680#0: *1202 http script copy: “”
30680#0: *1202 fastcgi param: “REDIRECT_STATUS: 200”
30680#0: *1202 http cleanup add: 09E3106C
30680#0: *1202 get rr peer, try: 1
30680#0: *1202 socket 13
30680#0: *1202 rtsig add connection: fd:13 signo:43
30680#0: *1202 connect to 127.0.0.1:10004, fd:13 #1203
30680#0: *1202 http upstream connect: -2
30680#0: *1202 event timer add: 13: 60000:3004683954
30680#0: rtsig add connection: fd:7 signo:43
30680#0: *1202 post event 09E7804C
30680#0: *1202 delete posted event 09E7804C
30680#0: *1202 http upstream send request handler
30680#0: *1202 http upstream send request
30680#0: *1202 chain writer buf fl:0 s:1272
30680#0: *1202 chain writer in: 09E31088
30680#0: *1202 writev: 1272
30680#0: *1202 chain writer out: 00000000
30680#0: *1202 event timer del: 13: 3004683954
30680#0: *1202 event timer add: 13: 60000:3004683955
30680#0: rtsig add connection: fd:7 signo:43
30680#0: *1202 post event 09E6B044
30680#0: *1202 delete posted event 09E6B044
30680#0: *1202 http upstream process header
30680#0: *1202 malloc: 09E31240:4096
30680#0: *1202 recv: fd:13 80 of 4096
30680#0: *1202 http fastcgi record byte: 01
30680#0: *1202 http fastcgi record byte: 06
30680#0: *1202 http fastcgi record byte: 00
30680#0: *1202 http fastcgi record byte: 01
30680#0: *1202 http fastcgi record byte: 00
30680#0: *1202 http fastcgi record byte: 34
30680#0: *1202 http fastcgi record byte: 04
30680#0: *1202 http fastcgi record byte: 00
30680#0: *1202 http fastcgi record length: 52
30680#0: *1202 http fastcgi parser: 0
30680#0: *1202 http fastcgi header: “X-Powered-By: PHP/5.2.5”
30680#0: *1202 http fastcgi parser: 0
30680#0: *1202 http fastcgi header: “Content-type: text/html”
30680#0: *1202 http fastcgi parser: 1
30680#0: *1202 http fastcgi header done
30680#0: *1202 malloc: 09E32248:4096
30680#0: *1202 HTTP/1.1 200 OK
30680#0: *1202 write new buf t:1 f:0 09E32268, pos 09E32268, size: 180
file: 0, size: 0
30680#0: *1202 http write filter: l:0 f:0 s:180
30680#0: *1202 http upstream process upstream
30680#0: *1202 pipe read upstream: 1
30680#0: *1202 pipe preread: 20
30680#0: *1202 readv: 1:4016
30680#0: *1202 pipe recv chain: 0
30680#0: *1202 pipe buf free s:0 t:1 f:0 09E31240, pos 09E3127C, size:
20 file: 0, size: 0
30680#0: *1202 http fastcgi record byte: 01
30680#0: *1202 http fastcgi record byte: 03
30680#0: *1202 http fastcgi record byte: 00
30680#0: *1202 http fastcgi record byte: 01
30680#0: *1202 http fastcgi record byte: 00
30680#0: *1202 http fastcgi record byte: 08
30680#0: *1202 http fastcgi record byte: 00
30680#0: *1202 http fastcgi record byte: 00
30680#0: *1202 http fastcgi record length: 8
30680#0: *1202 http fastcgi sent end request
30680#0: *1202 free: 09E31240
30680#0: *1202 pipe write downstream: 1
30680#0: *1202 pipe write downstream done
30680#0: *1202 event timer: 13, old: 3004683955, new: 3004683956
30680#0: *1202 http upstream exit: 00000000
30680#0: *1202 finalize http upstream request: 0
30680#0: *1202 finalize http fastcgi request
30680#0: *1202 free rr peer 1 0
30680#0: *1202 close http upstream connection: 13
30680#0: *1202 event timer del: 13: 3004683955
30680#0: *1202 rtsig del connection: fd:13
30680#0: *1202 http upstream temp fd: -1
30680#0: *1202 http output filter “/academy/80/test/12?”
30680#0: *1202 copy filter: “/academy/80/test/12?”
30680#0: *1202 http postpone filter “/academy/80/test/12?” BFFFBAA8
30680#0: *1202 http postpone filter out “/academy/80/test/12?”
30680#0: *1202 http chunk: 0
30680#0: *1202 write old buf t:1 f:0 09E32268, pos 09E32268, size: 180
file: 0, size: 0
30680#0: *1202 write new buf t:0 f:0 00000000, pos 0813310F, size: 5
file: 0, size: 0
30680#0: *1202 http write filter: l:1 f:0 s:185
30680#0: *1202 http write filter limit 0
30680#0: *1202 writev: 185
30680#0: *1202 http write filter 00000000
30680#0: *1202 copy filter: 0 “/academy/80/test/12?”
30680#0: *1202 http finalize request: 0, “/academy/80/test/12?”
30680#0: *1202 set http keepalive handler
30680#0: *1202 http close request
30680#0: *1202 http log handler
30680#0: *1202 free: 00000000
30680#0: *1202 free: 09E30238, unused: 36
30680#0: *1202 free: 09E32248, unused: 3358
30680#0: *1202 event timer add: 11: 5000:3004628956
30680#0: *1202 free: 09E2FBB0
30680#0: *1202 free: 09E2FE30
30680#0: *1202 hc free: 00000000 0
30680#0: *1202 hc busy: 00000000 0
30680#0: *1202 tcp_nodelay
30680#0: *1202 http keepalive handler
30680#0: *1202 malloc: 09E2FBB0:1024
30680#0: *1202 recv: fd:11 -1 of 1024

I’m perplexed. If it’s passing the correct stuff onto php, why isn’t it
running?

Thanks.

Hello!

On Sat, Mar 15, 2008 at 05:01:19PM -0400, Ian M. Evans wrote:

set $path_info “”;

  • if ($uri ~
    ^/(?:.+/)?(testgalleries|galleries|poll|news|photos|test)(/.*)) {
  • set $script_name /$1;
  • if ($uri ~
    ^(./(?:testgalleries|galleries|poll|news|photos|test))(/.)) {
  • set $script_name $1;

set $path_info $2;
fastcgi_pass 127.0.0.1:10004;
}

You old regex was capturing only last path component.

Maxim D.

Maxim D. wrote:

  • if ($uri ~
    ^/(?:.+/)?(testgalleries|galleries|poll|news|photos|test)(/.*)) {
  • set $script_name /$1;
  • if ($uri ~
    ^(./(?:testgalleries|galleries|poll|news|photos|test))(/.)) {
  • set $script_name $1;

    You old regex was capturing only last path component.

I think you’ve done it! All the test cases appear to work. $PHP_SELF
seems to not work as well in the CGI environment, but I’ve read about
issues with that in the past, so that’ll be a PHP fix.

Otherwise I think I’m pretty much ready to go 100% nginx.

Thanks!

Hi Ian,

Could you post the final and complete configuration file for the php
section so we can have a global look at it? That might save days of
work for other people in the future.

Glad you could sort it out 100%.

Maxim D. wrote:

[…]

30680#0: *1202 fastcgi param: “SCRIPT_FILENAME:
/usr/local/apache/htdocs/test”

Looks like you’ve got wrong SCRIPT_FILENAME. At least - this differs
from the one in the above request.

Thanks…that’s why a second set of eyes helps! I stared at that for a
while before posting and didn’t see it.

Looking at the location, can you see how it’s stripping the subdir(s)
out? Or how to get the subdirs back in?

Thanks!

location ~ /(testgalleries|galleries|poll|news|photos|test)(/|$) {
fastcgi_intercept_errors off;
error_page 404 /404test.php;
root /usr/local/apache/htdocs;
set $script_name $uri;
set $path_info “”;

if ($uri ~
^/(?:.+/)?(testgalleries|galleries|poll|news|photos|test)(/.*)) {
set $script_name /$1;
set $path_info $2;
fastcgi_pass 127.0.0.1:10004;
}

fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
fastcgi_param SCRIPT_FILENAME $document_root$script_name;
fastcgi_param PATH_INFO $path_info;
fastcgi_param REDIRECT_STATUS 200;
fastcgi_pass 127.0.0.1:10004;
}

Thomas wrote:

Could you post the final and complete configuration file for the php
section so we can have a global look at it? That might save days of
work for other people in the future.

Glad you could sort it out 100%.

Will do. The only thing I’m still noticing is the possible issue with
scripts that use $_SERVER[‘PHP_SELF’].

If a .php (or in my case, .shtml as well) file is called it equals the
filename e.g. _SERVER[“PHP_SELF”] = /test.php

If an extensionless file (in this case ‘test’) is called,
_SERVER[“PHP_SELF”] = no value

This is a bit of an issue for things like forms scripts or pagination
scripts that use _SERVER[“PHP_SELF”].

Obviously, I can rewrite my own stuff to not need _SERVER[“PHP_SELF”],
but for people who might look for a drop in solution or are using a
cut’n’paste script, it might be a problem.

If anyone has an idea why the extensionless files aren’t creating
_SERVER[“PHP_SELF”] or a way to create it, please let us know.

Thanks!