I’m not sure if this is on purpose, but it’s most likely that google is also moderating the biggest moderated directory on the net. Hehe that’s a good one.

See it yourself (I’m not sure for how long this links will be alive)
Dmoz link : updated around June 4th
Google’s Link: doesn’t exist as of June 5th

Also I’ve found if you search for url that exist on dmoz’s certain category, google returns a result but the link is simply for oh for.
See the first link.

Last notes: Yahoo’s directory doesn’t have the category too. I kind of understand it since the category is under Business branch and it’s a paid service for yahoo to appear under that branch.

Simply you add a line into your robots.txt saying

Sitemap: <sitemap_location></sitemap_location>

This part is really cool but for site harvesters it’s an unbelivable tool. So you can handover the key to your site and web harvesters can crawl your site really easily because probably you’ve put all your site’s pages into your sitemap.

Sounds like a good plan in an ideal world. With all the cloakers and content scrapers you must be really smart not to be ripped apart.

My suggestion is to know who you’re serving the sitemap. Currently Google, Yahoo and Ask is supporting this sitemaps.xml and no other site has anything to do with it.
Here is a simple check you can add in the begining of your sitemap thing:

< ?php
    function botIsAllowed($ip){
        //get the reverse dns of the ip.
        $host = strtolower(gethostbyaddr($ip));
        $botDomains = array('.inktomisearch.com',
                                     '.googlebot.com',
                                     '.ask.com',
                             );

        //search for the reverse dns matches the white list
        foreach($botDomains as $bot){
            if (strpos(strrev($host),strrev($bot))===0){
                $qip= gethostbyname($host);
                return ($qip==$ip);
            }
        }
        return false;
}

if (!botIsAllowed($_SERVER['REMOTE_ADDR'])){
    echo "Banned!";
    exit;
}
?>

I’m sure everyone can get the idea of reverse dns and forward dns checking.
If I missed any decent site that uses the sitemaps let me know.

Note: If you’re still using static sitemaps (!) you can just include the xml after the code.

These are the things to look out for when building a large application on web. Most of them I agree, some of them I don’t. This list also should keep in mind that, real world problems are not that perfect. I marked the ones that I strongly agree with an asterix (*).
I also filtered out the things are specific to del.icio.us from the original text.

• Browser quirks. CSS/JavaScript/rendering. IE caching.
• Scaling: avoid early optimization. SQL doesn’t map well to these problems – think about how to split up data over multiple machines. Understand indexing strategies, profile every SQL statement. Nagios or similar for monitoring.
• Tags don’t map well to SQL. Sometimes you can prune based on usage – only index the first few pages for example. This keeps indexes small and fast.
• *Some latency in the system is OK – work out where you have leeway, e.g. RSS feeds can fall a few minutes behind without anyone minding.
• *People are always going to abuse your system (scraping, greasemonkey, etc.)
• *“Idiots are a lot smarter than you” – wait to see what breaks before you fix it.
• *Learn Apache – tuning can make things a lot faster. Understand headers, mod_rewrite (a dark art). Put a proxy in front of Apache e.g. Perlbal from LiveJournal – one guy on a modem can suck up lots of resources otherwise.
• Images off a different server, RSS from a different server.
• “Save site for offline use” feature in IE is particularly nasty.
• *The easier the API to get in and out of, the more people will use it. There’s a long tail of smart developers. Stuff like SOAP discourages adoption. No API key = people can play with it faster; drives attention.
• Dont expose your unique id’s to the outside world (php?id=1 etc.) people can scrape through everything very easily. This is why del.icio.us uses MD5 hashes of links instead.
• Giving everything a unique ID in the database is a scaling problem in its own right.
• *Features: the features you put in are as important as the ones you leave out. There’s no “send a note” in del.icio.us feature because e-mail already exists.
• I don’t add featues that are available elsewhere e.g. messaging
• *When people ask for features, get to the bottom of why they are asking for that exact thing. Solve the problem, rather than doing exactly what your asked for.
• With tags, people ask for “A and B and NOT C or D” – but less than 1% of queries even use more than a single tag.
• *RSS important in del.icio.us, because it’s a native way for people to access lists (of links). Put RSS everywhere you can. del.icio.us does way more RSS traffic than HTML or API stuff – partly because of poorly written readers. Understand the headers – especially if-not-modified.
• *Make sure the URLs follow the path of the site. DONT include session data, drop ugly details that are to do with the system, not the user (.php, .aspx, ?, &, etc.)
• URLs are prime real estate – respect them (Good URL structure writeup from another carson speaker – http://www.plasticbag.org/archives/2003/06/on_permalinks_and_paradigms.shtml )
• *When you chose what to build, solve a problem you have yourself so you can be sure to understand it. Passion counts. It’s cheap and easy to build stuff, so other people will be building it too.
• *A niche product with a limited audience is still good business (with how advertising and paypal work)
• *Every day that you don’t have something properly out in the world (not on an invite only beta) you’re losing a chance to gain users. Get it out there ASAP.
• Aggregation is often a focus of attention (latest, most active, etc.)
• *As the population gets larger, the bias drifts; del.icio.us/popular becomes less interesting to the original community members. Work out ways to let the system fragment in to different areas of attention.
• “Spam is attention theft” – that’s one of the reasons del.icio.us doesn’t have a top 10 links of all time – it’s an attractive nuisance.
• *When you’ve figured out someone is spamming, don’t let them know – let them keep posting and just silently junk their stuff.
• *Make users do the minimum amount of work. But make them do something.
• Understand the user’s motivation: “You have to understand the selfish user” – user #1 has to find the system useful or you won’t get user #2. Systems that only become useful when lots of people are using them usually fail, because there’s no incentive for people to contribute themselves. The real trick is to make the user base you have want to invite more people in to the system.
• *Manage your effort – be careful where you spend your efforts.
• *Don’t waste time building features nobody uses.
• *Measurement: watch your system intensely: “Intuition is ideas backed by numbers”
• *Measure behavior rather than claims. del.icio.us doesn’t have stars because why would you bookmark something that was no good? This way people bookmark things that they really care about rather than trying to tell the system things.
• *User Acceptance Testing (UAT) is important. Make the system suitable for the people actually using the system. Everybody on the team should see this in action. Labs are great but expensive. If you don’t have a lab do ghetto testing in Starbucks. We did one day then two days user testing in a user testing lab.
• *Goals skew the results. People don’t read, they cram crap in to boxes. Let people wander don’t give them tasks.
• *You have to speak the user’s language. “Bookmarks” are what you call them if you use Netscape of Firefox – most users these days know the term “favourite” instead. Half of his population (? users) didn’t know what a bookmark was.
• *Don’t make users register before they can get in to your site. Maybe even give them an anonymous account to play with. A lot of users want to know what they’ll get if they register – especially from fear of giving out email address, spyware etc. You can’t tell them; they’re not going to read it. You have to show them.
• *Use Verbs – doing words – to prompt actions.
• *If users do have to register, send them straight back to where they were when they’re done. Don’t dump them on the homepage.
• *“Design Grammar” – if you’re presenting a system that’s different from how other things work (del.icio.us had novel tags, save your bookmarks to the web) you should still try to reflect the design patterns of the web.
• *Morals: You have to develop a sense of morals when you build your system. It’s the user’s data; it’s not yours. Make sure they can remove themselves and their account if they want to.
• *Infection: Understand infection vectors for promoting your system. “Enable evangelism”. RSS lets you get at users who don’t use your system directly. Also think about iCal, M3U – anything that a desktop app can consume over HTTP. Do an inventory to get into every desktop app. possible.

No AdSense Account
An AdSense account does not exist for this login, as you have not yet completed an application. If you are using your AdWords login, you can create an AdSense account with the same login.

I was really confused, I checked my emails an the bad news: Google AdSense Account Disabled.

“Due Invalid clicks, your google adsense account disabled”, is the email you receive and you don’t know what you’ve done wrong. Also the email reminds you that :

Publishers disabled for invalid click activity are not allowed further participation in AdSense and do not receive any further payment. The earnings on your account will be properly returned to the affected advertisers.

You can appeal to the account closure but it’s uncertain that they’ll enable you again. Google is really taking the invalid click issue seriously. I started searching the internet about other people getting disabled, to see if anyone has been able to get their accounts back.

But I was getting desperate after reading that people are kissing their accounts good bye after closures due to invalid clicks.

I replied to the email immediately asking why I am kicked out. There were no replys for the first day. So started changing all the adversitements on my site to other pay-per-click ads. Oh God no other PPC program pays as good as Google does. Why? Should I start worshiping Yahoo?.

I checked Yahoo’s PPC program but devastated by the notice: You must have a valid US SSN. Hey I’m a canadian can’t I participate Yahoo Ads?

I even changed my default search to Yahoo because I was so pissed off to Google.

The other day I received a brief email from Google Adsense saying “yeah you can appeal to this by sending an email to blah blah blah but it’s in our rights to accept you back or not.”

Good enough! I replyed the second email and waiting for an answers, 2 day passed.

By the time I realized that google was a really important part of my internet life. Switching to yahoo for search engine really sucked! It was really slow to do the searches, something that I haven’t realized before. I even stoped using Gmail, Google Groups etc. And I realized life without google is a real pain.

After the second day of appeal, I received another email from Google.  Hey they are taking me back!!! effective immediately and I’ll be saving my un-paid balance as well. Yes this is a good ending story. Maybe I’m the only one blogging about being taken back.(at least that I know)

I waited another 24 hour until the ad-display turns to normal.

In conclusion, in 3 days I realized how Google is important to me and to my website. I understand Google for the account closure, because there are 1000s of cheaters that they have to be careful about. But Google showed it’s spirit as concerning about my appeal. Google is really giant as a corporation but they are really keeping the amateur in spirit as they grow , concerning about each individual’s situations and etc.. That’s the difference between Micro$oft and Google I see. And I don’t think that Google will be a real big brother like everybody is affraid of.

Even if they become “the biggest” it’s no harm to the people like me. We should try to take benefit from that. But it’s a harm it’s competition and that’s too bad for them.

Anyway, loosing only 3 days of income, I’m back showing up the Adsense Ads. I strongly suggest you not to put all the eggs in the same basket, just in case. If you’re not a cheater and got a closure, don’t feel bad, explain your situation to google. Try to explain that your traffic is legit and you’re sensitive about the invalid clicks too.

They are in the end got bigger because of the people like you and me. They still care. At least by now.

 Did you like this story? Digg it here

Thoughts Aside:
The search engine submission is the most basic method to drive good traffic for your website. Plus email marketing enables your visitors to visit your every so often. Several network marketing tools are available on the internet that provides internet marketing solutions. And they can also inform you in case if Google disables your adwords advertising.

link

Yahoo Podcasting

I’m very my much amazed by the potential of this tool…
Also recently they are sold to Yahoo and Konfabulator is free…
Worths checking…

Yahoo Widgets
Konfabulator

THE fledgling R.S.S. business is starting to attract some attention from those catering to Internet advertisers.

Google, Pheedo, Feedster and Yahoo Search Marketing are all peddling advertising options for R.S.S., an increasingly popular way of having a personal computer automatically retrieve information from the Internet.

For example, R.S.S. users interested in local weather could view forecast updates on their computers without having to visit a particular Web site.

Some big companies, like Verizon, are starting to buy space in the R.S.S. information streams, which are selected anonymously and pulled from Web sites by a PC.

please read the rest of this article from NY Times