Skip to content

Latest commit

 

History

History
547 lines (373 loc) · 11.8 KB

README.rst

File metadata and controls

547 lines (373 loc) · 11.8 KB
Raw

vmod_parsereq

Varnish parse data module

Author: Syohei Tanaka(@xcir)
Date: 2013-06-25
Version: 0.12
Manual section:3

SYNOPSIS

import parsereq;

DESCRIPTION

SUPPORT DATA TYPES

  • POST
  • GET
  • COOKIE
  • REQ(req.http.*)
  • AUTO(auto type using for in subroutine only, that is called by iterate function.)

ATTENTION

Don't use "vcl.use" and "vcl.discard" on VCL with the vmod loaded. This will cause the a segfault.

FUNCTIONS

init

Prototype
init()

Parameter

Return value
void
Description

initialize.

Make sure to call parsereq.init() in vcl_recv before using any other function.

Example
import parsereq;

vcl_recv{
  parsereq.init();
}

setopt

Prototype
setopt(enum {enable_post_lookup})
Option
enable_post_lookup (EXPERIMENTAL)
 Enable to "CACHE" for POST request. very dangerous.
 This option modify to
  Backend request method set to "POST".
  Enable send body.
  Content-Length is set to POST size.
Return value
void
Description
Set option.
Example
import parsereq;

vcl_recv{
  parsereq.init();
}

errcode

Prototype
errcode()

Parameter

Return value
int errorcode

(sucess) 2   Unknown or missing content-type.
(sucess) 1   Parse complete.
(error)  -1  Out of session workspace. Try to increase sess_workspace
(error)  -2  No content-length key.
(error)  -3  Readable data < content-length.
Description
Get status code.
Example
import parsereq;

vcl_recv{
  if(parsereq.errcode()<1){

    ...

  }
}

param/post_header/get_header/cookie_header

Prototype
post_header(STRING key) (LEGACY)
get_header(STRING key) (LEGACY)
cookie_header(STRING key) (LEGACY)
param(enum {post, get, cookie, req, auto}, STRING key)
Parameter
STRING key
  Desired key value
Return value
STRING (not escaped)
Description
Get value.
Example
vcl_deliver{
  set resp.http.hoge = parsereq.param(post, "hoge");
}

//return
hoge: hogevalue

body/post_body/get_body/cookie_body

Prototype
post_body() (LEGACY)
get_body() (LEGACY)
cookie_body() (LEGACY)
body(enum {post, get, cookie})

Parameter

Return value
STRING (NOT ESCAPED)
Description

Get (get,post,cookie) raw data.

This function is dangerous. The raw data is not escaped. Usage of this require a thorough understanding of the risks involved.

Example
//vcl
vcl_deliver{
  set resp.http.hoge = parsereq.body(post);
}

//return
hoge: hoge=hogevalue&mage=magevalue

next_key/post_read_keylist/get_read_keylist/cookie_read_keylist

Prototype
post_read_keylist() (LEGACY)
get_read_keylist() (LEGACY)
cookie_read_keylist() (LEGACY)
next_key(enum {post, get, cookie, req, auto})

Parameter

Return value
STRING
Description
get (get,post,cookie) key name.
Example
//req
/?name1=a&name2=b

//vcl
vcl_deliver{
  set resp.http.n1 = parsereq.next_key(get);
  set resp.http.n2 = parsereq.next_key(get);
  //nothing
  set resp.http.n3 = parsereq.next_key(get);
}

//return
n1: name2
n2: name1

reset_offset/post_seek_reset/get_seek_reset/cookie_seek_reset

Prototype
post_seek_reset() (LEGACY)
get_seek_reset() (LEGACY)
cookie_seek_reset() (LEGACY)
reset_offset(enum {post, get, cookie, req, auto})

Parameter

Return value
VOID
Description
Reset the seek index.
Example
//req
/?name1=a&name2=b

//vcl
vcl_deliver{
  set resp.http.n1 = parsereq.next_key(get);
  set resp.http.n2 = parsereq.next_key(get);
  parsereq.reset_offset(get);
  set resp.http.n3 = parsereq.next_key(get);
  set resp.http.n4 = parsereq.next_key(get);
  //nothing
  set resp.http.n5 = parsereq.next_key(get);
}

//return
n1: name2
n2: name1
n3: name2
n4: name1

size

Prototype
size(enum {post, get, cookie, req, auto}, STRING key)
Parameter
STRING key
  Desired key value
Return value
INT
Description
Get the size of value.
Example
//req
/?name1=a&name2=bbb

//vcl
vcl_deliver{
  set resp.http.n1 = parsereq.size(get, "name1");
  set resp.http.n2 = parsereq.size(get, "name2");
  //nothing
  set resp.http.na = parsereq.size(get, "name99");
}

//return
n1: 1
n2: 3
na: 0

current_key

Prototype
current_key(enum {post, get, cookie, req, auto})

Parameter

Return value
STRING
Description
Get current key-name of the offset.
Example
//req
/?name1=a&name2=bbb

//vcl
vcl_deliver{
  set resp.http.t1 = ">>" + parsereq.current_key(get);
  parsereq.next_offset(get);
  set resp.http.t2 = ">>" + parsereq.current_key(get);
  parsereq.next_offset(get);
  set resp.http.t3 = ">>" + parsereq.current_key(get);
  parsereq.next_offset(get);
  set resp.http.t4 = ">>" + parsereq.current_key(get);
}

//return
t1: >>
t2: >>name2
t3: >>name1
t4: >>name1

next_offset

Prototype
next_offset(enum {post, get, cookie, req, auto})

Parameter

Return value
VOID
Description
Change to the next key. If next key isn't exist, will not change.
Example
//req
/?name1=a&name2=bbb

//vcl
vcl_deliver{
  set resp.http.t1 = ">>" + parsereq.current_key(get);
  parsereq.next_offset(get);
  set resp.http.t2 = ">>" + parsereq.current_key(get);
  parsereq.next_offset(get);
  set resp.http.t3 = ">>" + parsereq.current_key(get);
  parsereq.next_offset(get);
  set resp.http.t4 = ">>" + parsereq.current_key(get);
}

//return
t1: >>
t2: >>name2
t3: >>name1
t4: >>name1

iterate(EXPERIMENTAL)

Prototype
iterate(enum {post, get, cookie, req}, STRING)
Parameter
STRING subroutine pointer
Return value
BOOL
Description
Count all elements in parameter for iterate the subroutine. This function is subject to change without notice.
Example
//req
/?name1=a&name2=bbb

//vcl
sub iterate {
  set req.http.hoge = req.http.hoge + parsereq.current_key(auto) + ":";
  set req.http.hoge = req.http.hoge + parsereq.param(auto, parsereq.current_key(auto)) + " ";
}
sub vcl_recv {
  parsereq.init();
  if(1 == 0){
    call iterate;
  }
  set req.http.hoge= "";
  C{
    if(Vmod_Func_parsereq.iterate(sp, "get", (const char*)VGC_function_iterate)) return(1);
  }C

}
sub vcl_deliver{
  set resp.http.t1 = req.http.hoge;
}

//return
t1: name2:bbb name1:a

INSTALLATION

Installation requires a Varnish source tree.

Usage:

./autogen.sh
./configure VARNISHSRC=DIR [VMODDIR=DIR]

VARNISHSRC is the directory of the Varnish source tree for which to compile your vmod. Both the VARNISHSRC and VARNISHSRC/include will be added to the include search paths for your module.

Optionally you can also set the vmod install directory by adding VMODDIR=DIR (defaults to the pkg-config discovered directory from your Varnish installation).

Make targets:

  • make - builds the vmod
  • make install - installs your vmod in VMODDIR
  • make check - runs the unit tests in src/tests/*.vtc

Trouble shooting

You could try to increase the sess_workspace and http_req_size parameters and stack size(ulimit -s).

Tested Version

  • 3.0.1(x86_64)
  • 3.0.2(x86_64)
  • 3.0.2-streaming(x86_64)
  • 3.0.3(x86, x86_64)
  • 3.0.4(x86_64)

HISTORY

Version 0.12: Support POST request cache(issue #7 thanks nshahzad). 3.0.4 support(issue #8)

Version 0.11: Support REQ data type.(req.http.*) And AUTO data type.

Version 0.10: Add: param, size, body, next_key, next_offset, current_key, iterate, reset_offset

Version 0.9: Bug fix: always segfault on x86. And sometimes segfault on x86_64. [issue #5 Thanks comotion]

Version 0.8: Support unknown content-type.(post_body only) [issue #3 Thanks c0ze]

Version 0.7: Bug fix: forgot to care binary. [issue #4 Thanks dnewhall]

Version 0.6: Bug fix: when you vcl reloaded, hook method be off.

Version 0.5: Rename module(parsepost -> parsereq)

Version 0.4: Add get keylist function.

Version 0.3: Support GET,COOKIE, modify interface.

Version 0.2: Rename module(postparse -> parsepost)

Version 0.1: Add function parse

COPYRIGHT

This document is licensed under the same license as the libvmod-rewrite project. See LICENSE for details.

  • Copyright (c) 2012 Syohei Tanaka(@xcir)

File layout and configuration based on libvmod-example

  • Copyright (c) 2011 Varnish Software AS

parse method based on VFW( https://github.com/scarpellini/VFW )

url encode method based on http://d.hatena.ne.jp/hibinotatsuya/20091128/1259404695