My Anti-virus Program Comparison Analysis

[Wai_Wai]

My Anti-virus Program Comparison Analysis

Version 0.3

Note:

- Again this post becomes more and more long-winded. So scan the bold/italic headings first. If the heading interests you, read on.

- But please read this heading " Important!!!!! You should know before reading my reviews!!!!! " before going straight to my reviews.

==================================

Update logs:

v0.3

Another 1/2 volume of contents are added

- make more clarifications about this post, my comments and my reviews.

- explain more about the limitations of my reviews and the summary tables.

- inclusion of Important!!!!! You should know before reading my table!!!!! So hopefully readers will not be misled when reading my tables.

- clarify, add some comments made on anti-virus programs

- The dates of the reports are added

- mistakes on the images are discovered

v0.2

A lot of contents are added. The post expands by 3/4 volume.

- Explanation on My evaluation methodology

- Add a lot more comments on the anti-virus programs

- Have some overviews on their strong and weak points

- Includes links to my reports

- Includes links to other brief reports

v0.1

- the first draft of my post

==================================

Introduction

It is hard for us to judge if an anti-virus can protect us well. Simply using it cannot give you accurate evaluation. Consider this case. A virus bypassed your anti-virus program. They don't cause serious problems in your computer. You never notice of its existence. You still feel your anti-virus program is doing a great job.

In this regard, I did a long research. Afterward I tried to pick up some of the best anti-virus programs and introduce to you. So you can save your time and troubles reading a lot of reports, or thinking hard on picking a good AV program.

I tried to summarize a lot of reports and provide one large table for users to read. In the table, I try to compare different anti-virus programs in terms of their virus protection capabilities, including a lot of areas, namely:

- ITW (known/common virus) protection

- Zoo (unknown/rare virus) protection

- heuristic system

- false positives (ie false alarm to clean files)

- compressed files

- non-viral malware protection

- office infected files

- virus removal ability

- and so on

Hopefully you may find the information useful. Enjoy!

==================================================

My evaluation methodology

I am going to comment on their anti-virus capabilities based on the reports I have read, some of which brief, some detailed; and some of the tests/trials I made (if applicable).

I value detailed reports much more than simple/brief reports because they analyse their anti-virus capabilities in a comprehensive and thorough way, not just focus on one aspect only. Usually the brief reports assess their abilities to detect known (ITW) viruses. And it is not uncommon for them to make mistakes.

I hardly rely on magazine reviews because their analyses are light and may be partial as some articles point these problems out.

I would pick up the best ones if more reports (at least 2, and especially the detailed ones) rank them excellent. I will not conclude an anti-virus program as excellent just by 1 report says so.

Even within the same report, some parts may not be representative enough (eg because I notice their sampling size is small). Then I will not rely a lot on these results in making my judgement.

I won't really add any subjective points/opinions into my judgement. I am based on the results given from reports and tests, and make conclusions.

Finally I have provided a table which summarises most of the detailed reports. The brief reports are excluded. But I have provided links, so you can read them yourself.

All the summary tables and links can be found at the end of my post.

Help yourself and enjoy!

Important!!!!! You should know before reading my reviews!!!!!

Since some readers are (or will be) frequently raising such kinds of questions/challenges, it is the best to make a good strong emphasis first. So reader will not be misled from the information in my post.

Q: So are you suggesting XXX is the best AV program, and all people must use this one?!? I'm no longer a 3-year-old baby!!!

A: No, I haven't made such claim in anywhere of my post. I just pick the best AV program based on their AV capabilities only, and no more. High AV capabilities are just one aspect (although important). There are some other aspects which we haven't considered (eg features, ease of use, compatibility/stability). You may need to consider them as well before making a decision.

In a nutshell, AV with the best AV capabilities is never a byword for the best AV!! Don't be confused with these 2 ;-)

Q: Your comments and recommendations are extremely subjective! Please consider rewriting it.

A: All my comments are based on rock solid facts, I try my best to isolate all my sentiments before reach my judgement. I don't add any comments/opinions/points which are not found in the reports.

And I don't rely on ONE SINGLE report to make my judgement. This is to prevent the mistakes, bias or whatever bad things made by a report. In fact, I've read a lot of reports in order to reach the conclusions.

In fact, my spirit was as if on vacation when I was writing my report. I completely submitted to what the reports say. All are written by the reports, NOT me. Scary huh?

If you ever find one single point which is not concluded from any report, please tell me and I will gladly remove it.

Q: Your conclusions are completely unacceptable. They are all contradictory to our common sense. XXX is known to be the best. Everyone knows except you id***.

A: Bear in mind, if I say something is not good, it is in terms of their AV capabilities and no more. What's more, it is the reports which lead me to the conclusions, NOT me. My spirit was away when making such judgement.

As far as AV capabilities is concerned, if it is said their AV capabilities are not good enough, I am confident to tell you it is very likely to be the case.

I realise it is exceptionally hard to accept. But it is painfully true. It is the same to me. When I see how my favorite AV programs score poorly, I feel upset. I don't wish to accept the truth and comfort myself, saying such-and-such reports must be mistaken and so on. But it is not just 1 report which says so, at least it has to be 2 very reliable sources in order to make me such kinds of conclusions.

However there are some limitations in the reports. For details, see the heading " Limitations" at the end of my post.

Q: How can you say XXX is abysmal? In fact there are much more crappy AV programs which are worse than XXX. Why don't you criticise them? You are too demanding!!!!!

A: All anti-virus programs which are short-listed should meet the general standard. Otherwise I will not list them in the first place.

In fact, all are about comparisons. All comments are relative. XXX is said to be bad if others are better than XXX. When others only detect 50% of virus and XXX detects 70%, it is already the best and we will say it is excellent. However when others detect 90% but XXX detects 70% as usual, we will no longer say XXX is good anymore. It is because the standard is pushing up.

The same case holds true again. But among the top products, they are just bad by comparison. In fact, most of them do good jobs.

Q: Why don't you analyse more AV programs like YYY or ZZZ? There are far far more anti-virus programs in the market. Do you have any evil plans in mind?

A: Yes, I only shortlisted the well-known & good ones. It is because most people wish to know about them. It appears to be no point in spending time on analysing a crappy or immature AV program, just to tell you how crappy the AV program is. And I doubt people care to know about that.

However it is worth analysing among all good AV programs, so you can see their strong and weak points by the process of competitions.

By the way, I may have some evil plans in mind. Who knows (including me)? ;-D

The best anti-virus programs

[Note: All comments are based on the information found in the reports. I don't add any personal statements/opinions in making my judgement]

McAfee http://www.mcafee.com/us/?cid=10550

- A well-rounded anti-virus(AV) program which achieve well in most of its areas, but not perfect. (Anyway no AV program is perfect)

- It is the only program which can remove ALL viruses (100%) successfully in a series of virus removal tests performed by a report.

- It hardly generates any false positive which is a merit. 0 false positive is impressive.

- It has problems in detecting virus in archived and compressed files though.

Kaspersky AVP http://www.kaspersky.com/

- It focuses a lot on its detection capabilities. It can catch more viruses than others (eg Norton).

- It does well to catch unknown viruses too which is also an aspect we should not ignore.

- It works harder to deal with non-viral (less harmful) malware which other anti-virus programs often ignore.

- But it can't disinfect well.

- In a test, it has serious problems in catching any boot virus. Yes, it caught 0 boot virus when you access to the infected files.

- It generates some false positives once in a while. McAfee can generate none for most of the time. It can't.

Seemingly good AV programs

The following may be good although I would like to read more reports to confirm:

F-Secure http://www.f-secure.com/

- It seems it incorporates multi-search engines into its anti-virus program, but one article argues that it doesn't help you much. It is just a marginal benefit. The costs don't outweigh the small benefits.

- Anyway, it seems to have good virus protection although I need to read more to confirm.

AVK http://www.antiviruslab.com/

- This program seems good but only gets German version only.

- I haven't included this in my analysis (because the program is German) . More reading is needed to confirm its quality.

Some other anti-virus programs

It may surprise you much, and exceptionally hard to accept. Some anti-virus programs are well-known but do not do their job well. They just can't beat the best ones:

Norton Anti-Virus (Symantec)

- Although it is a long-established anti-virus company, I couldn't imagine it can score poorly in some areas, as if it were an immature new anti-virus program. I suppose it is excellent, at least in terms of anti-virus protection.

- It cannot detect and scan well as most people might suppose so

- It has problems in scanning archived/compressed files

PC-cillin (TrendMicro)

- Doesn't do well in detecting both known and unknown viruses.

- can hardly equal McAfee and Kaspersky in terms of anti-virus capabilities.

Avast (Alwil)

AVG (Grisoft)

- quite many people recommend these 2, but unfortunately here's the bad news - their AV shields are not strong, I'm afraid. They are not mature at this stage.

- They can't catch known viruses well. Avast (80.55%); AVG (72%). At least it needs to be above 90% in order to meet the case.

- They can become infirm in face of unknown viruses.

- They can't handle archived/compressed files properly.

- Generate far more false positives than Norton and McAfee.

NOD32 (Eset)

- As a comparison, it is better in detecting unknown viruses than known viruses.

- But it is not a good idea since the chance of encountering a known virus is much higher than that of unknown.

- Need to work hard to deal with known viruses. It scores 82.68% only in one test. Fail!

- False positives are one of the problems

Panda Anti-Virus

- Don't protect well.

- Become infirm in face of unknown viruses.

- System crashed in WinME while scanning in one of the test!

- Have some glitches.

================================================================

About my table

I would like to say sorry first of all.

The table is far from perfect. I haven't explained each entry and their scores. I rely on your wisdom to interpret the data, still less it is too simple and ugly.

But that substandard table has already spent me 1 day to produce it! Unbelievable but true!

I slept late at 4:00am on that day, and had to wake up early at 8:00am on the next day to work... Exhausted...

If you don't understand some parts of the table and would like to know more, ask me and I will explain to you.

Even if you just wish to know more details about your favorite AV programs, you may ask me too, and I will compile more for you. Alternatively, you may read the links and explore yourself.

My table is just a starting point to give you some general ideas about your favorite AV programs.

Details of analysis

Back to the issue, the tables are as follows:

[Note: Thanks for telling by a kindhearted user. I made a silly mistake at the company name of RAV (GeACD). Please read GeACD as GeCAD. Blame me for making a table at midnight!!]

Annual Report 1

(Date of the report: 2004. See, very new and up-to-date report! )

http://img74.exs.cx/img74/9296/avcompare017ol.gif

Annual Report 2

(Date of the report: 1st test - 2002; 2nd test - 2001)

http://img72.exs.cx/img72/3131/avcompare021hx.gif

Annual Report 3

(Date of the report: 1st test - 2003; 2nd test - 2002)

http://img98.exs.cx/img98/9688/avcompare035tj.gif

Thanks for ImageShack http://reg.imageshack.us/v_images.php for free image hosting.

Limitations

Outdated reports are the problems.

I have to admit the reports I chosen are not up to date. Most of the detailed reports are 1-2 years ago. Av-comparatives produce the latest reports. But frankly, they reports are not as comprehensive as some of my other reports (although it is still better than quite many other reports/reviews).

But why detailed reports are always outdated? It is easy to understand why. It's because a good and comprehensive anti-virus report needs a lot of time to produce - Half year is not unbelievable! It is never impossible to finish a report within a month unless you are going to read some magazine reviews.

I rely on detailed reports to make most of my comments because they are more trustworthy and reliable than brief reports and magazine reviews. But the price is I cannot get up-to-date information.

Think twice, if the information is not accurate or reliable, what's the point of getting them even if they are up-to-date?

How to deal with this 1-year gap?

Nevertheless we don't really need to worry too much about this limitation (the information is 1 year old. I call it 1-year gap ). If the normal situation goes, a good program will keep being good even after 1 year. If you haven't heard of any (major) bad news from the AV program within the year, it is quite safe to assume the program is still good. It shouldn't change dramatically in this 1-year gap.

On the other hand, if you hear from many magazines saying some new anti-virus programs do very well (or they suddenly improve substantially) in this 1-year gap, but the detailed reports are not available, it could be a painful dilemma. However I would like to say something about magazine reviews (or its similar types):

- Most simply do not have enough resources to conduct effective and representative anti-virus capability test. Unless the magazine is using the results from a big and independent testing organisation, the reviews cannot reflect their true value.

- Some magazines receive money support from these anti-virus programs (by advertisements etc.) So do you think they are will be impartial enough

- Small magazines may rely on analyses or research data from big magazines. Then they make their reviews and comments based on these data. So...

But many users praise anti-virus programs highly. So it must be good, right? Yes, it may be. But I would like to point out some of the cases where it would not be true:

- Users comments are based on the magazine reviews they have read. And magazine reviews are actually... so...

- Experiences may lie unfortunately. Consider this case. A virus bypassed your anti-virus program. They don't cause serious problems in your computer. You never notice of its existence. You still feel your anti-virus program is doing a great job.

- An anti-virus program generated a false positive, falsely claiming that the file is infected. You think it is great. Other anti-virus programs cannot detect this virus, but this anti-virus program can. Excellent!

Finally, I wish you good luck on the road towards the best anti-virus program.

Other links of brief reports:

http://www.virus.gr/english/fullxml/default.asp

http://www.virusbtn.com/

http://www.icsalabs.com/

Excellent sources of anti-virus comparison reports!!

http://www.msfn.org/board/index.php?showto...=0entry242685

[Wai_Wai]

Updated

[glent]

Very useful thanks

[Wai_Wai]

Very useful thanks

Thanks that you find them useful.

If you have any enquiries relating my post or even the AV programs (eg you may wish to ask why some rpograms perform badly etc.), free feel to ask me.

[Spyder2k]

Excellent post, should finally help me decide which AV or combination of AVs to use. Very informative thankx.

[lajes]

very informative, thank you for your time and hard work.

[horsecharles]

Awesome work Wai_Wai-- actually awesome's an understatement: any publication should be willing to pay good bucks for such a piece.... Actually you do do that already likely....?

I'd be curious of your opinions on the AV in Etrust, the suite with ZA & Pest Patrol in it? The AV used to be standalone-- Dr. something(Solomon maybe, also perhaps started out in the Mac platform originally...)

Thank you.