Joomla’s non-optimized search in content.searchbot.php

Hi, All!

Recently I faced a MySQL replication lag time problem. From time to time it increased dramatically (up to 200 sec’s) on new boxes (there are 4 boxes in cluster beyond load balancer) with 4Gb of RAM and really-really nice CPUs. I had to enable slow queries logging to figure out what was causing the problem.

Okay, the slowest query was…. yeah, in search subsystem...

Read more

OpenSEF RC5 SP2 Perfomance Improvement

Hi, all!

Recently I’ve rewritten a pretty large site on Joomla (it has about 200 online users all the time, as Joomla backend stats shows, >17000 content items etc.) and noticed that the page generation time started to grow ftom 0.101 secs to 0.5 after the first day, 0.7 - 1 next day…). Not very good, right? First of all I went to the MySQL console at the time of high loads and… voila - there is a `opensef_sef` MySQL query present almost constantly. Guys, can anybody explain me why do they fetch into the memory almost all records in sef.php [Line #173]:

if ($sefSite->id == null) $sefSite->id = -1;
$query = “SELECT * FROM #__opensef_sef WHERE published = ‘1′” .
“\nAND valid = ‘1′” .
“\nAND (direction IS NULL OR direction = ‘’ OR direction = ‘o’)” .
“\nAND (site_id IS NULL ” .
($sefSite->id ? ‘ OR site_id = ‘ . $sefSite->id : ‘’) . ‘)’ .
“\nORDER BY site_id DESC, published DESC, valid DESC, LENGTH(direction) DESC”;
$database->setQuery( $query );
if ($database->loadResult()) {
$opensef_aliases = $database->loadObjectList();
}

A quick fix is to replace the query with this one:

if ($sefSite->id == null) $sefSite->id = -1;
$query = “SELECT * FROM #__opensef_sef WHERE published = ‘1′” .
“\nAND valid = ‘1′” .
“\nAND (direction IS NULL OR direction = ‘’ OR direction = ‘o’)” .
“\nAND (site_id IS NULL ” .
($sefSite->id ? ‘ OR site_id = ‘ . $sefSite->id : ‘’) . ‘)’ .
“\nAND (external=’{$_SERVER[’REQUEST_URI’]}’” .
“\nORDER BY site_id DESC, published DESC, valid DESC, LENGTH(direction) DESC”;
$database->setQuery( $query );
if ($database->loadResult()) {
$opensef_aliases = $database->loadObjectList();
}

After that, find the lookupOutgoing function  in administrator/components/com_sef/sef.class.php and replace it with this one:

function lookupOutgoing(  $site_id, $url ) { // static method
       global $opensef_aliases, $database;

       $returnaliases = array();
/*
       if ($site_id == null) $site_id = -1;
       if (count($opensef_aliases) > 0)        {
       foreach($opensef_aliases as $aliases){
                       if ( $aliases->internal == $url ) {
                               $returnaliases[] = $aliases;
                       }
               }
       }
*/
     if (!count($returnaliases))
     {
         $query = "SELECT * FROM #__opensef_sef WHERE published = '1'" .
         "\nAND valid = '1'" .
         "\nAND (direction IS NULL OR direction = '' OR direction = 'o')" .
         "\nAND internal='$url'".
         "\nORDER BY site_id DESC, published DESC, valid DESC, LENGTH(direction) DESC limit 1";

         $database->setQuery( $query );
         if ($database->loadResult())
         {
           $aliases = $database->loadObjectList();
           foreach($aliases as $k=>$val)
           {
             $returnaliases[] = $val;
           }
         }
     }

     if (is_array( $returnaliases ) && count( $returnaliases )) {
     $returnaliases = xclCastObjectList( $returnaliases, 'JosOpenSEFAlias' );
     return $returnaliases[0];
   }
   $null = null;
   return $null;
 }

Hurray! It works as a charm. Page generation time is now 0.07 seconds.

Hope this will help somebody else.

Thanks.

Adsensers or "What's next?"

This is just a copy of my post. Original here.

Have you evere thought about web without advertisement? Don't you think it's possible? May be yes, may be no. But we have to give it a try...

Few years ago I was surprised seeing somebodies project with 3 blocks of google-like ads. "What's that?", - I thought, "Where is the content?". Affiliate links and text ads made up to 80% of the site. It was unusual for 2001-2002 years. But now thousands of web-projects turned from good nice looking source of interesting news etc. into ugly monsters where one should search for the info he came for. And there is a tendence: as more popular is website as more banners it has. Ads on the top, ads on the left, right, bottom and even inside the content! That's ugly! Pretty project becomes to look like a dirty chicken.

Yes, I understand that is a way to earn some money. But guys, why don't you try to sell your content itself instead of selling your luck? "May be somebody will click the links and I'll get a cent... or even dollar!". People, please try to forget about that. I see more and more adsense-fishing sites without any sort of own content. With such a tendency our children have good chance to get web without something outdated earlier know as "good content". Even now advertising of advertising is not astonishing.

Why am I writing this? I don't expect wellknown bloggers and webmasters to remove superfluous. But I hope somebody (even if he'll be one of a thousand) will read the post and decide to leave only one adsense block on the site instead of putting it everywhere.

Thanks.

Moving to .com domain

Hi, all. I'm moving from blogger.com to my .com domain, so mainly posts will be published there. I'll try to keep this one posted, but if you're interested change bookmarks to custompcsoftware.com.

Joomla again.

Hi, all!

I was happy to realize people are reading my article about Joomla and even refer to it. Just FYI: I'm working on the second part of that post, hope you'll be interested in it too.

For now I'd like to ask Joomla and Opera users whether I'm the only who experience high processor load while viewing some of Joomla sites like Joomla! Developer network. Move your mouse over the page -- and the processor load jumps up to 100%. I use Win2k sp4 and Opera 9.00. Any ideas?

Thanks.

Joomla vs Drupal?

Recently I was asked to write a review for CMS of best-choice. It's full text is below, so you can check and leave comments.

What CMS to choose?

Well, let’s begin from the beginning. What are the most possible candidates? Drupal (http://drupal.org), Joomla (http://joomla.org) and Wordpress (http://wordpress.org – YES! I’m not mistaken, WordPress can be used as CMS). These projects beat anything else I know.

I’d like to begin from the Wordpress. The subject is well-known as blogging engine, but only a few people know it can be used as CMS. Here is what John McCreesh wrote at ONLamp.com:

Could WordPress be used for full-blown websites as well as blogs? I believe the answer is definitely yes. WordPress lets users do an awful lot of things without requiring any knowledge of the underlying technology, such as maintaining content, including images; creating categories for the content; selecting the look-and-feel of the site from a large and growing range of templates; managing multiple users with different access rights; and adding and removing plugins for extra functionality. For users with a working knowledge of web technologies such as HTML, CSS, or PHP, then so much the better -- they can dive under the bonnet/hood (depending on which side of the Atlantic you dive from) and work all sorts of magic. The end result is the best of both CMS and blog worlds, with flexible content layout arranged under hierarchical menus such as a CMS, plus all the nice features of blogs such as ease of maintenance, RSS feeds, comments, and permalinks.

As you can see it’s possible to use WordPress as a web-project engine. If you have content-driven project and you do not need e-commerce integration, your business logic is straight-through and simple – WordPress is right for you. A lot of plugins for RSS, polling etc. will help you to build the website quickly and easily.

But what if we need a sort of complex site? Among hundreds of CMSs I distinguish Drupal and Joomla. There is a lot of reasons why these and not other projects, you can check all of them yourself if you don’t trust me. To be honest the only other project that’s worth seeing is Xoops (http://xoops.org). But it’s neither user-friendly nor geniously coded (it is great coded though!) so I’m leaving xoops alone.

Let’s compare those monsters. Monsters in a good meaning. The first thing that caught my eye is Joomla’s admin panel – take a look at http://demo.joomla.org/demo10/administrator/ (alfim/joomlapass). It’s completely user-friendly, isn’t it? Here is what xaneon.com says (Mambo is an older sister of Joomla):

Mambo is certainly “easy on the eyes”: most people react very favorably to seeing the administration interface for the first time. Another aspect where Mambo is definitely ahead of the game is in installation friendliness, as well as add-on management (installation and uninstallation of components, modules, etc.)
In comparison, Drupal requires one to manually unzip add-ons on the server, possibly create the necessary SQL tables from supplied scripts, and there is no friendly installation “wizard” to guide you through first-time installation. (This is all evolving, though; it shouldn’t be many months before there is a comprehensive installation system available in Drupal.)
If the above points are very important considerations to you, as they certainly might be to less technically-savvy users, then you may not really benefit from this article. As stated, our team hails from a quite different user segment.

And I completely agree with that. Also, Joomla has thousands of third-party components and templates (much more than Drupal has). That’s the first reason why I’m in love with Joomla.

I hope you know what is Google Trends. Shortly: it’s a tool that analyzes Google web searches and that can visualize search trends over time using so called "search-volume graphs". These graphs usually provide a good mechanism to compare the popularity of two or more products. Buytaert.net provides us with the search-volume graph that compares Drupal and Joomla:

It is worth pointing out that Joomla has been around a lot longer than the graph suggests. In 2005, the bulk of Mambo's core developers left Mambo and started Joomla after a dispute with Miro Corporation, the company that founded Mambo. Keep this in mind when interpreting the graph. (I tried adding Mambo to the graph but the term Mambo isn't unique to Mambo, the content management system.)
That said, you can see that Joomla is more popular than Drupal, and that Joomla has been growing a lot faster. Why? The general consensus is that Joomla has a more appealing balance between functionality, flexibility, performance, quality of code, ease of use, documentation, user interface design, support and product marketing.

And that’s the second reason why I recommend Joomla – it’s popular, it has very… very big community so all bugs are found and fixed quickly and number of different add-ons for that CMS grows extremely fast.
You might have heard Mambo* is “bulky, badly optimized for search engines, and generally rigid and brittle to customized. Drupal, on the other hand, is perhaps the most search engine friendly CMS on the market. Its modular, flexible, its underlying design has been guided by a stellar philosophy” (http://www.nicklewis.org/mambo-vs-drupal) . But Mambo is not a Joomla (check Nick's comment below).

*There was "Joomla" before Nov 15, thanks to Nick Lewis for correction.

Now, to be honest, I’d like to tell you there is one thing why I can recommend you to use Drupal. It’s coded better than Joomla. It’s a bit faster than Joomla. Buytaert.net says Drupal is 319% faster than Joomla (wow!) but that’s also not a reality. To check this I took my old PC (P-133Mhz) installed there LAMP (Linux+Apache+MySQL+PHP), set up Joomla and Drupal, modified code a bit to get perfomance data and after all of that I got interesting things – Joomla and Drupal have shown comparable time. Drupal was faster only 1%-3%. Old PC demonstrated server’s behaviour at high loads, so … that’s the third reason why I still love Joomla.

And, to get this finished, here’s what users wrote at Yahoo! Answers:

There are many Open Source CMS applications and as usual choosing an appropriate one is difficult. In my experience, I find Joomla and Mambo the best but I prefer Joomla as it is being developed more at present as it is an off-shoot of Mambo.
Drupal looks impressive but I found it hard to configure. We used another one called Midgard for a project but there has been little development of that CMS plus it is hard to configure.
The reasons why I like Joomla better are as follows:
1. Joomla has many extensions (add-ons) so it can be scaled easily - http://extensions.joomla.org
2. There are many free as well as paid templates available.
3. Joomla is search engine friendly and this is an important consideration while marketing your website on the Internet. There is as an Search Engine Friendly (SEF) extension at http://open-sef.org/
4. Most hosting providers have Joomla that can be installed through their Control Panels.

So, if you are a hard programmer, you like to experiment with code, you don’t care much about user-friendly and usability or you think Drupal’s solution is OK – use Drupal.

If you care about your customers, you wish to ba able to set up addons (components and modules) with 2-3 mouse clicks without even unpacking files manually on server (Joomla will do that for you) or you’re just like me love Joomla – go ahead and give it a try.

Added on Nov, 16th: Guys, I'm not the only who respect Joomla. PackTrub admitted it to be the best. Wow!

Alexander Alfimov. Nov, 09 2006

А в это время...

Давно я что-то писал, а если что-то и писал, то удалял :)

В этот раз вдохновили комментарии на http://alexclear.livejournal.com/350021.html#comments

Пока все там хаят PHP, большое количество серъезных организаций в штатах уходят от Java и ASP.NET на PHP. Обратите внимание, именно на PHP, а не на что-либо иное. Хочу поговорить с менеджером одной из таких компаний, узнать, что именно их побудило перейти на сабж. Так что, обещаю в скором времени интересный пост :).

А пока, оставьте, пожалуйста, комментарии, если у кого-то есть свое мнение по этому поводу.

Спасибо.