Language Selection

English French German Italian Portuguese Spanish

New Content/Layout OK?

Varnish Proxy

Silly me, a poll would not work on the new server. I forgot that with the Varnish cache proxy at the front almost all visitors arrive from the same IP address (the proxy), which means that Drupal would allocate just one vote to all (except registered and presently logged in users). With Drupal upgrade we can perhaps find polling software that overcomes this.

rpaf

You must use mod_rpaf to fix this problem that Varnish introduces.
See eg https://www.varnish-cache.org/lists/pipermail/varnish-misc/2008-September/016470.html
mod_rpaf for EL6 64bit here: http://centos.alt.ru/repository/centos/6/x86_64/mod_rpaf-0.6-2.el6.x86_64.rpm

Proxy

Thank, we will look into it. Currently, a lot of stuff other than the poll (e.g. views being counted) are not compatible with Varnish and it makes it look as though not many people visit and can participate in the site.

For sheer stats you could use

For sheer stats you could use an external (i.e. not cached by varnish) service, such as Google Analytics or run your own Piwik.

Piwik

Google Analytics is spyware, but Piwik would be a possibility (Stallman recently told me that it's good). Can it be installed on a cache proxy? I'd have to gain access to it first. Either way, this would not facilitate per-post page request count. Susan had it set up with a module, but it's no longer working correctly. In turn, rating/sorting posts by popularity is no longer possible, and that's the real downside (the front page can no longer list popular items for today).

The problem is not just that IP addresses are not unique. Some requests are never seen by the CMS and Apache.

For the non-unique addresses

For the non-unique addresses look at mod_rpaf, it was made for this situations.
Is this drupal6 or 7? With 6 varnish integration sucks from what I've seen.

See also
https://drupal.org/project/varnish
https://fourkitchens.atlassian.net/wiki/display/TECH/Configure+Varnish+3+for+Drupal+7

Agreed on Google Analytics. You can just install Piwik on the same host and tell Varnish either not to cache it or you can just set its virtualhost on a port other than 80 so it bypasses Varnish completely.

Varnish

Thanks for the pointers.

Yes, it's Drupal 6 and there are other issues that I am beginning to see, such as lack of updates from the RSS feeds around the page (I am currently investigating this, maybe it's related to a cron job or module config although I very much doubt the latter as I haven't changed configs).

Non-unique addresses could be bypassed as an issue even by writing random IP addresses, but that would enable easy poll rigging. I guess it's not essential for operation of the site, but it's a nice-to-have...

From Drupal.org: "This module provides integration between your Drupal site and the Varnish HTTP Accelerator, an advanced and very fast reverse-proxy system. Basically, Varnish handles serving static files and anonymous page-views for your site much faster and at higher volumes than Apache, in the neighborhood of 3000 requests per second."

I have had such issues with Varnish on top of WordPress and MediaWiki (pages served improperly from cache) and it all makes me wonder if removing Varnish altogether is the best way to proceed.

As for Piwik, I have never tried it before, so I will look into it.

I would keep Varnish on for

I would keep Varnish on for static files (css, js, jpeg etc) and to clean up HTTP traffic (Varnish will not forward incomplete or malformed HTTP requests to the backend, it should also be the front line against synfloods etc).

Here's a sample of what I use (test it first, I'm just beginning with Varnish myself)

director default dns {
.list = {
.port = "8080";
.connect_timeout = 5s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
.max_connections = 10000;
"172.16.1.53"/32;
}
}
sub vcl_recv {
if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
return(lookup);
}
}
sub vcl_fetch {
if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
unset beresp.http.set-cookie;
}
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
}

Then install mod_rpaf and make sure your Apache is listening on port 8080 and add this to /etc/httpd/conf.d/rpaf.conf:
LoadModule rpaf_module modules/mod_rpaf-2.0.so

RPAFenable On
RPAFproxy_ips 127.0.0.1 IPs_OF_THE_SERVER
RPAFsethostname On
RPAFheader X-Forwarded-For

PS: looks like drupal is messing with my comments, here's a text version http://fpaste.org/74672/raw/

Thanks

Thanks, I will look at it and into it in the weekend.

RSS feeds

The Piwik demo looks impressive, I have just given them a word of endorsement.

I am still trying to resolve some other issues we've identified.

I think I found the source of the issue above (RSS feeds). It seems like any external site access is denied by default, which helps explain why RSS feeds cannot be retrieved by the Drupal part of the site:


[root@tuxmachines ~]# wget lxer.com
--2014-02-05 04:34:37--  http://lxer.com/
Resolving lxer.com... 108.166.170.174
Connecting to lxer.com|108.166.170.174|:80... failed: Connection refused.
[root@tuxmachines ~]# wget linuxtoday.com
--2014-02-05 04:34:54--  http://linuxtoday.com/
Resolving linuxtoday.com... 70.42.23.121
Connecting to linuxtoday.com|70.42.23.121|:80... failed: Connection refused.

Looks like a firewall issue

Looks like a firewall issue at the first glance.

Firewall

Nux wrote:

Looks like a firewall issue at the first glance.

Yes, it was a simply issue to tackle. It works now.

Pageview count and polls

I'll have a look and see if configuration can solve not just the polling issue but also pageview count. The site of this module is down and it seems like it may require configuration on the cache server too.

More in Tux Machines

Linux 5.4 To Expose What's Keeping The System Awake Via Sysfs

The next Linux kernel version will expose the real-time sources of what's keeping the system awake via Sysfs compared to existing source information that previously was only available via DebugFS. With Linux 5.4, wakeup source statistics will now be exposed under /sys/class/wakeup/wakeup*/ for tracking wakeups, their counts, and related bits for profiling the system for power usage optimizations. Suspend-blocking wakeup sources have been available under DebugFS to be useful for bug reporting and analyzing battery consumption. This solidifies the work now under Sysfs with a stable ABI. In addition to the interfaces now stable in adding them to sysfs, it makes them more accessible compared to DebugFS often being restricted to root or other restrictions in place by different distribution kernels. Read more

Chrome murders FTP like Jeffrey Epstein

What is it with these people? Why can't things that are working be allowed to still go on working? (Blah blah insecure blah blah unused blah blah maintenance blah blah web everything.) This leaves an interesting situation where Google has, in its very own search index, HTML pages served by FTP its own browser won't be able to view... Read more

Programming Leftovers

  • Creating a Docker Swarm Stack with Terraform (Terrascript Python), Persistent Volumes and Dynamic HAProxy.

    Before someone blame me about why I am not using Kubernetes, AWS ECS, Mesos or anything but Swarm the answer is simple: Docker Swarm is an inexpensive and very simple thin orchestrator. Because of this it miss a lot of features that Kubernetes already implemented by default. Most of important data centers (Google, AWS, Azure, Oracle, IBM, Digital Ocean, etc) already implemented some sort of Kubernetes as a Service make it easy its adoption. However, docker swarm does not have any datacenter are implementing it and are creating some of structure ready to go as K8s has.

  • Python Filter()

    Python filter() function applies another function on a given iterable (List/String/Dictionary, etc.) to test which of its item to keep or discard. In simple words, it filters the ones that don’t pass the test and returns the rest as a filter object. The filter object is of the iterable type. It retains those elements which the function passed by returning True. We can also convert it to List or Tuple or other types using their factory functions. In this tutorial, you’ll learn how to use the filter() function with different types of sequences. Also, you can refer to the examples that we’ve added to bring clarity.

  • Sending HTML messages with Net::XMPP (Perl)

    This started with a very simple need: wanting to improve the notifications I’m receiving from various sources.

  • Excellent Free Books to Master Programming

    A quick search of the internet reveals a plethora of books for programmers. No one has time to read even a minuscule fraction of the available books. What you need is a curated list of programming books. Better than that. A curated list of free programming books. Free and open source books still have a cost — your time. And just because a book is free/open source doesn’t, itself, signify any great quality to the work. Hence the need for some recommendations for free books to help you learn C, C++, Java, Python, R, or whatever language takes your fancy. The books we’re recommending will help increase your technical skills and make you proficient in the language of your choice. And some of them even provide a little light relief on the way. Humor can be a great aid to learning.

Anime studio, Khara, is planning to use open-source Blender software

The open-source 3D creation software, Blender, will be picked up by the Japanese anime studio, Khara. It’ll begin partially using the software for its current development ‘EVANGELION:3.0+1.0’ but will make the full switch once that project is finished. The current project is expected to end in June next year, so after that point, its employees will start using Blender for the majority of their work. At the moment, Khara uses 3ds Max from Autodesk on a subscription basis; however, the company found that it had to reach out to small and medium-sized businesses for its projects. Due to the limitations of those companies, it’s harder for them to afford 3ds Max. By switching to Blender, Khara says it can work better with external firms. Read more