Tag Archives: website

Strange HTTP GET requests from IP 150.70.x.x and IP 62.24.x.x

Recently I started to notice that one of our sites gets strange HTTP GET requests from two IP ranges: 150.70.x.x and IP 62.24.x.x.

The short version of this goes as follows: we have web service when users can submit results via HTTP GET request. For example:
http://www.example.com/service?guid={BEDC2C9A-C5E6-4766-B57C-7CC07BB26F59}&result=x

Each (next) user/request gets a new GUID, like:
http://www.example.com/service?guid={412E9B56-E9CE-4FB5-9804-676FDC9EA3EC}&result=a
http://www.example.com/service?guid={64A00D1D-BCDB-4B5A-9DF3-51CF2BB6B663}&result=b
http://www.example.com/service?guid={CA5B86A1-A6B4-4CA8-AAA6-DC9302CF34D3}&result=c

Because GUIDs are unique, we should get each GUID only once. But this was not true for our web-service. We occasionally got duplicate requests. We started to investigate closer, and found that all double requests come from two IP ranges 150.70.x.x and IP 62.24.x.x, and all “second” requests follows the first after 30s – 5 min.

We did a Google search and found, that 150.70 IP range belongs to Trend Micro. There are even some statement from them:

Dear Site Owner,

To protect our customers from visiting a malicious or harmful site, web pages of the applicable URLs are downloaded and scanned by our servers. Thus, you may have noticed a few visits from our IPs. Please be assured that this poses no security risk to your web sites as our servers do not perform any action other than scanning the sites.
We then store the rating of the web site in our server cache so that our servers will no longer access those pages for analysis when a customer chooses to visit those web sites again. We have already asked our server owner to add the rating for the following domain(s);
…..
The setting will take effect soon, please verify it again on your site.
Sorry for any inconvenience and please inform us of any other concerns you may have.
Best Regards,
Trend Micro’

More: http://forum.notebookreview.com/security-anti-virus-software/331930-trendmicro-servers-japan-accessing-webservers.html

It seems, that Trend Micro is spying after their customers. Second range comes from Talk Talk UK’s ISP, who apparently spies after their customers too.

Here are some links with more reading:
Search Engine Spider and User Agent Identification Forum from webmasterworld.com
IP Address Inspector – projecthoneypot.org
Im Being Monitored/Watched?

P.S. Most probably they do not try to do HTTP POST request, I do not know. We probably should use POST too, but that’s another story.
P.S.S. Some encryption like SSL should be mandatory for all Internet traffic in future.

Things that will make your website much faster

What will make my website faster? How to optimize page load time? How to make web site appear to load faster for end user?

Updated Feb 9, 2011.

This is excerpt from article: How big HTML images / CSS / JS / png / etc. should be to fit network packet (MTU) efficiently

List of things that will make your website much faster:

  • Server must support Keep-Alive. Otherwise change server / host / hosting company / etc. Do it NOW!
  • Reduce HTTP requests (CSS sprites, combine CSS, combine JS, inline CSS, inline JS)
  • Minifiy HTML, JS, CSS. Google Closure Tools, YUI Compressor, Minification
  • Gzip text/html, css, js (IE6 does not un-gzip CSS and JS, if reference is not from HEAD)
  • For IE 5 & 6 use gzip-only-text/html
  • Check if (Content-Encoding: gzip) then (Vary: Accept-Encoding) to allow cache both versions in proxy servers
  • CSS goes in head, JS goes at bottom – right before body closing tag
  • Defer JS if possible. Defer allows to load js after onload
  • Image formats: for photos – always use JPG, for everything else – PNG 8, for animations – GIF
  • Compress images: for JPG use jpegtran or jpegoptim Adobe Photoshop or Advanced JPEG Compressor (both non-free) or free GIMP, for PNG use OptiPNG or PNGOUT. I use non-free but the best one – PNGOUTWin
  • Think connections! Waterfall. webpagetest.org (probably the best site for web page speed / optimization test), Zoompf (a little overkill, but can be useful), Pingdom Tools
  • Try to serve assets in parallel, e.g, images.example.com, images2.example.com; subdomains can be on the same IP
  • Use 2-3 max 4 sub-domains
  • Optimize response headers, smaller, meaningful
  • Avoid redirects (301, 302, META refresh, document.location)
  • Remove ETAG
  • Use Expires + cache-control
  • 25k and greater files are not cached on iPhone
  • Cache dynamic content PHP, ASPX, ruby etc.
  • example.com?param is not cached by most proxies
  • Serve static assets from cookie-less domain, like yimg.yahoo.com
  • After onload via js can pre-cache images (if you know where visitor will go next)
  • Can use double heads (if a lot of meta then put 2nd head at the bottom after closing BODY tag)
  • Use CDN if can afford

And remember about:

Related tools / links:

More reading and watching:

While making website faster, do not forget about security:

Things that big guys (such as Google, Microsoft, Yahoo, Akamai) do. This requires custom software, customized Linux / BSD / Apache compilation, and of course deep knowledge in networking, OSI model, TCP, etc.:

P.S. This blog (wishmesh.com) is not optimized for anything (speed, size, etc.). People are lazy. They optimize only when there is reason to do it, and we like most humans are lazy, so this blog runs on default WordPress hosted on shared server.

How big HTML images / CSS / JS / png / etc. should be to fit network packet (MTU) efficiently

List of little things that will make your website much faster.

Not every webmaster knows, that 1 extra byte in jpg image, can cause 100% speed reduction / penalty for your website visitor. When you learn that MTU for Ethernet is 1500 bytes with actual data / payload size of 1460 bytes, you can calculate optimal image / resource sizes.

First network packet gets penalty because it includes HTTP server response. Typical HTTP 1.1 response is from 284 for PNG up to 363 for JS, but usually is bigger if not optimized. You can remove most of stuff in this response, but you will get another penalty in form of missed caches that leads to slower website loading speed for user with repeat views (returning visitors).

Here is an example of good optimized HTTP response header for PNG image:
HTTP/1.1 200 OK
Date: Tue, 19 May 2010 14:07:55 GMT
Server: Apache
Last-Modified: Wed, 05 May 2010 14:43:16 GMT
Content-Length: 2502
Cache-Control: public
Expires: Thu, 18 Jun 2010 14:07:55 GMT
Keep-Alive: timeout=5, max=99
Connection: Keep-Alive
Content-Type: image/png
[CR][LF]
...data goes here...

If you like to mess with Apache you can remove Server header, but there is a known bug, that makes this removing non-trivial.

Making response as small as possible, makes room for more actual payload (be it – jpg, png, css, js) in first network packet. So, optimally image / script / stylesheet file size needs to occupy at most bytes of used packets. For example, if your image is 1300 bytes, then payload size is 1300 + 284 HTTP response, thus first packet payload is full 1460 bytes and second packet is used inefficiently – only 124 bytes. As we know, we send packets not bytes or bits. There is little to no difference in sending one byte or 1460 bytes. In this example, the suggestion is to reduce image size by 124+ bytes, thus causing image to fit into one packet. Or use 1336 bytes to increase image quality, thus causing image to fit in the same 2 packets, but with two times better image quality. (Anyway, these 1336 bytes is almost always unused / wasted)

Now if you know your header sizes and you optimize your site for best user experience / speed, it is wise to use all space provided by packets.

It is easy to create Spreadsheet with recommended payload sizes for all resources using the following formula:
1460 * network packets - HTTP response = maximum resource size

Remember, that every resource (png, jpg, gif, js, css, etc.) will have different HTTP response header size.

Here is a snippet from table I use daily for one of my websites that needs to be optimized for fastest page loading speed:
Optimize payload size for png

If you exceed resource by one byte, you immediately get one extra packet, that can cause speed reduction by 100%.

Now if you now this, you can go and examine www.google.com or www.yahoo.com root pages. You will find that all images are optimally fitted in optimized packet count / sizes. For example google.com png response size is 278 bytes, image is 6803 bytes. So Google is using approximately 1200 bytes of last packet. It is not coincidence, it is designed with page loading speed / quality in mind.

Also, remember that this is only small step in web site speed optimization. Here are some other tips from my checklist that can cause significant increase of page load times / user experience.

List of little things that will make your website much faster:

  • Server must support Keep-Alive. Otherwise change server / host / hosting company / etc. Do it NOW!
  • Reduce HTTP requests (CSS sprites, combine CSS, combine JS, inline CSS, inline JS)
  • Minifiy HTML, JS, CSS. Google Closure Tools, YUI Compressor, Minification
  • Gzip text/html, css, js (IE6 does not un-gzip CSS and JS, if reference is not from HEAD)
  • For IE 5 & 6 use gzip-only-text/html
  • Check if (Content-Encoding: gzip) then (Vary: Accept-Encoding) to allow cache both versions in proxy servers
  • CSS goes in head, JS goes at bottom – right before body closing tag
  • Defer JS if possible. Defer allows to load js after onload
  • Compress images (for photo like always jpg, for everything else PNG 8, for anim GIF)
  • Think connections! Waterfall. webpagetest.org (probably the best site for web page speed / optimization test), Zoompf (a little overkill, but can be useful), Pingdom Tools
  • Try to serve assets in parallel, e.g, images.example.com, images2.example.com; subdomains can be on the same IP
  • Use 2-3 max 4 sub-domains
  • Optimize response headers, smaller, meaningful
  • Remove ETAG
  • Use Expires + cache-control
  • 25k and greater files are not cached on iPhone
  • Cache dynamic content PHP, ASPX, ruby etc.
  • example.com?param is not cached by most proxies
  • Serve static assets from cookie-less domain, like yimg.yahoo.com
  • After onload via js can pre-cache images (if you know where visitor will go next)
  • Can use double heads (if a lot of meta then put 2nd head at the bottom after closing BODY tag)
  • Use CDN if can afford

And remember about:

P.S. To cover each optimization topic in details, I can write a whole book. I am not going to do this yet, so if you have any questions, ask in comments!

Author Ha.

Updated 22 Jun 2010.