Modify request body before sending to upstream

Server NGINX
1

Hi,

I am new to nginx module development and I’m working on my first ever
module. I’ve read up Evan M.'s post besides others, and I’ve
experimented tweaking some simple modules.

From what I understand, proxy_pass module is a handler and we can
effectively have just one handler run on a request. What I need is to do
some work with the content before I send a request to the upstream
servers.
I have been able to achieve the reverse via filter modules, but not
this. Is
there an way to achieve this without touching proxy_pass?

The requirement comes from a server rewrite we are doing to improve
performance. We have nginx load balancing requests to a bunch of servers
running python. We decided to rewrite some of the python pre-processing
in
C/C++ and write an nginx module to wrap around it.

Please lead me the right way :).

Abhishek

Posted at Nginx Forum:

2

Hi,

Any info on this? I’ve been trying a ton of ways to achieve this but it
seems like I’m really lost.

To repeat with clarity, I need to operate on the request body first,
modify
it and THEN send it off to upstream servers with the modified content.

Any pointers must help. Please :slight_smile:

Abhishek

Posted at Nginx Forum:

3

Hi Abhishek,

i’m not 100% sure to understand exactrly what you’d like to do,
especially the request-body-manipulation-part.

nginx_lua is usually quite handy when you have the need
to manipulate a request:

you can jump into the acces- or rewrite-phase, make your processing and
pass the result to your upstream-servers using proxy_pass and all the
upstream {} - goodies

cheers,

mex

Posted at Nginx Forum:

4

Thanks!

I tried ngx_lua but I might’ve been doing something wrong. It complained
that I am not allowed to use “proxy_pass” following a content rewrite.

To make it even simpler, here’s a simplified example:

  • curl -X POST --data “ABCD” localhost:8080
  • an NGINX module that calls a custom C function to alter the string,
    say
    “a[1]+=5”, so now we have “AGCD”
  • send “AGBC” to upstream app
  • respond with whatever the upstream responds (no filters beyond this)

Posted at Nginx Forum:

5

Hi,

I tried ngx_lua but I might’ve been doing something wrong. It
complained that I am not allowed to use “proxy_pass” following a
content rewrite.

you should read the documentatrion carefully:

“Do not use this directive and other content handler directives in the
same
location.
For example, this directive and the proxy_pass directive should not be
used
in the same location.”

what you can do is use the access_by_lua or rewrite_by_lua - directive

cheers,

mex

Posted at Nginx Forum:

6

Nevermind my previous post. I solved it finally :slight_smile:

location /foo {
    rewrite_by_lua '
        res = ngx.location.capture("/bar", {method = ngx.HTTP_POST,

body
= “jjj”})
res = ngx.location.capture(“/bar”, {method = ngx.HTTP_POST,
body
= res.body})
ngx.print(res.body)
';
}

location /bar {
    proxy_pass http://wservers;
}

[vm ~]$ curl localhost:8080/bar -X POST --data ‘hello’
UPSTREAM: hello :UPSTREAM
[vm ~]$ curl localhost:8080/foo -X POST --data ‘hello’
UPSTREAM: UPSTREAM: jjj :UPSTREAM :UPSTREAM

Thank you so much!! You saved me a lot of time.

Posted at Nginx Forum:

7

Still no luck. Here’s my config:

upstream wservers {
server localhost:8001 max_fails=3 fail_timeout=2s weight=100;
server localhost:8002 max_fails=3 fail_timeout=2s weight=100;
}

server {
location /foo {
rewrite_by_lua ’
ngx.print(“yay”)
';
proxy_pass http://wservers;
}

location /bar {
    proxy_pass http://wservers;
}

}

Here are the curl commands:

[vm ~]$ curl localhost:8080/bar -X POST --data ‘hello’
UPSTREAM: hello :UPSTREAM
[vm ~]$ curl localhost:8080/foo -X POST --data ‘hello’
yay

What I need is for the second curl command to output:
UPSTREAM: yay :UPSTREAM

Posted at Nginx Forum: