Questions tagged [esi]

Edge Side Includes or ESI is a small markup language for edge level dynamic web content assembly. The purpose of ESI is to tackle the problem of web infrastructure scaling by inserting ESI include tags directly into cached HTML documents. A ESI processors analyses the document to support different caching mechanisms and times different parts of the document, allowing for increased personalisation of cached pages.

It is fairly common for websites to have generated content. It could be because of changing content like catalogs or forums, or because of personalization. This creates a problem for caching systems. To overcome this problem a group of companies (Akamai, Art Technology Group, BEA Systems, Circadence Corporation, Digital Island, Inc., Interwoven, Inc., Open Market, whose ESI-related technology is now owned by FatWire Software, Oracle Corporation and Vignette Corporation) developed the ESI specification and submitted it to the W3C for approval. The proposal editor was Mark Nottingham.

ESI Language Specification 1.0 was submitted to the World Wide Web Consortium (W3C) for approval in August 2001. The W3C has acknowledged receipt, but has not accepted the proposal.

ESI is implemented by some content delivery networks, such as Akamai, and by some caching proxy servers such as Varnish, Squid and Mongrel ESI, although many do not implement the complete specification. Akamai also adds additional features to the version they support.

How ESI is implemented

ESI element tags are inserted into HTML or other text based content during creation. Instead of being displayed to viewers, these ESI tags are directives that instruct an ESI processor to take some action. The XML based ESI tags indicate to the edge-side processing agent the action that needs to be taken to complete the page's assembly. One simple example of an ESI element is the include tag which is used to include content external to the page. An ESI include tag placed in-line within an HTML document would look like this:

<esi:include src="http://example.com/1.html" alt="http://bak.example.com/2.html" onerror="continue"/>

In this case the ESI processor would retrieve the src URL, or failing that the alt URL, or if that failed do nothing. The ESI system is usually a caching proxy server so it may have a local copy of these files which it can insert without going back to the server. Alternatively the whole page with the ESI tags may be cached, and only the ESI requests may be made to the origin server. This allows different caching times for different parts of the page, or different degrees of personalisation.

Features

There are four main features in ESI:

  • inclusion of page fragments, as illustrated above;
  • variables which can be set from cookies or HTTP headers and then used in other ESI statements or written into markup;
  • conditions so that different markup can be used based on variables, for example if a cookie is set or not;
  • error handling, so that a failover can be used if an origin server is unavailable.
112 questions
20
votes
1 answer

How to structure a Symfony2 app with ESI?

On a new project with lot of traffic, we are thinking on how to structure our Symfony2 app to take advantage of caches, and be ready to be more aggressive in the future. I'd love to know your opinion. Let's say a user requests a page a list of…
fesja
  • 3,313
  • 6
  • 30
  • 42
9
votes
2 answers

ESI includes in JSON response

I'm struggling getting ESI includes to be replaced in a JSON context. The following working correctly. Resulting in the ESI tag replaced by the actual contents. And this is not working. The ESI tag…
Bram Gerritsen
  • 7,178
  • 4
  • 35
  • 45
9
votes
1 answer

Are ESI blocking or non-blocking?

I'm planning my caching strategy and I'm digging into ESI to see if it fits my needs. Today these questions arose: Is each include executed in serial or parallel? Say I have 5 of these: {% render '...' with {}, {'standalone': true} %}. I understand…
ChocoDeveloper
  • 14,160
  • 26
  • 79
  • 117
6
votes
1 answer

Incrementing a Page View Count with Varnish and ESI

If I am using Varnish to cache my entire documents, by what mechanism would you advise I increment a page view count as well. For example, lets supose that I have an auction listing, such as ebay, and I would like to cache the entire page since I…
Layke
  • 51,422
  • 11
  • 85
  • 111
5
votes
1 answer

Edge Side Includes and validation cache in Symfony 2

Is it possible to use validation cache in an ESI with Symfony 2 ? If you look the HttpFoundation Response class, you can see how isNotModified works: /** * Determines if the Response validators (ETag, Last-Modified) match * a conditional value…
Sybio
  • 8,565
  • 3
  • 44
  • 53
5
votes
0 answers

ESI caching in Symfony 3

I have enabled ESI in my Symfony 3 app and I have this in app_dev.php: $kernel = new AppKernel('dev', true); $kernel = new AppCache($kernel); Now I have config.yml: framework: esi: { enabled: true } fragments: { path: /_fragment } In my…
KondukterCRO
  • 543
  • 2
  • 16
  • 31
5
votes
1 answer

How do I invalidate cache for a controller url?

I want to invalidate the HTTP cache in symfony2. I use the following method: protected function invalidateCache($url) { $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PURGE'); …
Tobias Nyholm
  • 1,142
  • 9
  • 16
5
votes
1 answer

Symfony ESI breaks Varnish cache if cookies are enabled

I don't know what I am doing anymore. I had so many issues I don't know where to start. Here is my configuration: varnishd (varnish-3.0.3 revision 9e6a70f) Server version: Apache/2.2.22 (Unix) Symfony 2.3.1 First I've disabled Symfony AppCache in…
maxwell2022
  • 2,818
  • 5
  • 41
  • 60
4
votes
1 answer

Symfony2 homepage HTTP cache validation and independently ESI

I'm designing a project which has the same structure as a blog with Symfony2. My home displays articles and have a sidebar where there are links to login, or links to our account if we are logged. My sidebar is an ESI, my question: If I set a…
Sybio
  • 8,565
  • 3
  • 44
  • 53
4
votes
1 answer

Symfony2, Varnish and ESI lead to strange behavior

I have the following config: varnish (80) <-> nginx (8080) <-> php-fpm (9000) (Same behavior using Apache with mod_php) My Varnish config: backend default { .host = "127.0.0.1"; .port = "8080"; .connect_timeout = 600s; …
prehfeldt
  • 2,184
  • 2
  • 28
  • 48
4
votes
0 answers

Make varnish cache a synthetic response for failing ESI

We use varnish 4 for caching and processing ESI. For ESI requests that return a 5XX backend error, we return an empty synthetic response. That empty response should be cached for a few seconds, because currently every subsequent request to the same…
simon.ro
  • 2,984
  • 2
  • 22
  • 36
4
votes
1 answer

Any communication between parent request and ESI sub requests (varnish, session needs)?

I'm trying to work out varnish. My problem is. I do have a main request/response which I want to cache using varnish. Inside html structure a have ESI tag. I want to have many of them. One tag with longer ttls and others with ttl=0s; (ESI content…
4
votes
1 answer

Is caching using Edge Side Includes a good practice for a lot of small blocks?

We have a lot of very small blocks that we want to include with an ESI in an Symfony2 application. To be concrete, it is about rating stars that get displayed in a list of products and about an user avatar and rank which gets displayed in a list of…
Maximilian Ruta
  • 518
  • 7
  • 18
4
votes
1 answer

How to cache dynamic content in varnish?

I am working on an RoR app. I want to set a ttl for sold out items. The sold_out_status function returns the productIds of of sold-out products and is defined as: def sold_out_status query_str = params[:pids] pids = query_str.split(",") sold_outs…
nish
  • 6,952
  • 18
  • 74
  • 128
4
votes
1 answer

Can the result of an esi:include or esi:eval statement be stored in an esi variable?

I would like to store the result of an ESI include or eval block in an ESI variable. The contents of the include is outside of my control. Its format is JSON. I've tried all of the following to no avail. The variable ends up as an empty…
ringzhz
  • 81
  • 6
1
2 3 4 5 6 7 8