Summary
Short are the days of the Wild West on the Internet. This phenomena is taking on the characteristics of the US west’s history- namely the appearance of the digital farmers who are treating their new found digital assets as investments to be maintained and hopefully productive.
This is happening because of the way Google search algorithms determine the best results for each search term. While many still feel the internet is a new youngster, it is soon to be adult at 21 years, and with its maturity many of the wild west open range characteristics of the cowboy days are giving way to the land grab of the farmers.
Search terms are finite, and as such they are dominated by those legacy websites that recognized the potential early, and have invested accordingly, meaning new websites will have a tough time competing.
Author Biography
Ron Burgess was born and raised in Colorado. He attended the University of Colorado where he carefully manipulated matriculation, receiving two separate undergraduate degrees; a BA in Fine Arts and BS in Business.
While working to meet student expenses he worked at a local college retail specialty shop, where he was fortunate to combine the realities of small business with the theory of academic big business education. This lead to an opportunity to open a retail store one week after graduation. Seven stores were opened over ten years.
In the early 1980’s Ron accepted a position with a national retail consulting firm where he was responsible for new product development and training of 120 field consultants. His early involvement with the PC and spreadsheets, somehow was thought (in those days) to be training for software development. Despite this misconception, he managed to finish a multimillion dollar software conversion from an IBM mainframe to an IBM 386 PC, due to the brilliance of his technical team. The PC applications sold like hotcake for $22,500 each.
Despite Ron’s ignorance about software development, this experience did eventually lead him to the Internet. A decade after opening his own business consulting shop in 1989, he started RedFusion with son Jon and wife Molly, to build websites and support his clients in technical marketing issues of the day. This company became the largest web developer in the Inland Empire during the Wild West years of the Internet’s youth. Today the company manages the entire marketing process for dozens of smaller corporations, and is proficient at the emerging technology of online marketing and digital media.
Ron and wife Molly have three grown sons, who together with their wives have created 5 grandchildren, all of whom seem to know more about this smart phone than he does.
Fortnightly Club
Ron Burgess
Google: Fencing In the Wild West-
Why new websites will never get top search listings
The days of website equal opportunity on Google may be over. New website and blog content has new hurtles to getting top search results.
It was already a decade ago when I presented to you my paper on “Accessing Knowledge From Clay Tablets to Google.” At the time the Internet was a young teenager, in a few months it turns 21, an adult, still with much life in front of it, but beyond the gangly and uncoordinated years of its youth.
Many developments have enhanced our lives, and now provide instant access to so much knowledge that it is said that if one year’s information were printed on books, and they were stacked up, they could form a new wall of china each year. [ sic] jon **need stat
We enjoy the Internet using Wi-Fi connections through-out out the day, from home, to our cars, to the Y for morning workouts, our offices, and restaurants for lunch and finally here in the Smiley Library. Of course we don’t need Wi-Fi if we have a smart phone, because they bring us the Internet everywhere we go; even as they track our whereabouts using GPS.
Much has happened online since my first paper. Today the Internet is the undisputed king of technology, as the PC business has become mature and smart phones and tablets are mostly successful based on connectivity to the net. Billionaires have been created with small investments, and few people, in a new kind of non-capital capitalism.
The near future brings innovations under the label of “The Internet of Things” which are quickly connecting our home thermostats to the Internet, security systems to our smart phones, and thousands of companies are working quickly to create millions of “things” that will communicate continually with other things connected to the Internet. The physical will soon be transmitted through the Internet in digital form and printed one at a time on existing 3D digital printers in 30 materials from ceramic to metal. Despite these developments the majority of organizations, choose to keep a toe in the water instead of swimming.
When I wrote “Accessing Knowledge From Clay Tablets to Google,” I was optimistic about how Google would provide a level playing ground for smaller businesses looking for exposure to a broader market. I later published several articles on the subject of the Internet’s Wild West, a fence free, expanse of open territory, just waiting for the business to start herding digital long horns to the market, and win financial rewards for their hard work.
Google through its superior search engine tools, was helping businesses of all sizes get found by indexing the websites, selecting the key-words, and presenting pages to searchers with uncanny accuracy. Ambitious institutions, businesses, non-profits, and even individuals could carve out their own narrow niche for anyone who wanted information about that niche.
My optimism about a website’s free access to online searchers, continued for over a decade. Like Horace Greeley, shouting “Go west young man,” we shouted “Go online small business.” Of course it may have seemed like a self-serving mantra, but at the core, we root for the small business individualism and entrepreneurship that for 400 years has created the new products and markets that we so enjoy as part of our standard of living today.
But the slow moving trends indicate that now, just like the open prairie of the cowboy, the Internet is becoming fenced in by virtual farmers using Google search as digital bobbed wire. While the Internet is nearly infinite in its ability to add more information, the key terms we use to find that information is in fact finite.
There is a limit to the kinds of information most of us look up. While these terms number in the hundreds of thousands, or millions, some of these terms are already “owned” in the sense that certain web pages have become the defacto experts.
Let’s recall some of the lessons of ten years ago about how Google works:
“Page and Brin, the Google founders, reasoned that the millions of web authors, who dealt in the subject matter where they were experts, created hyperlinks from one of their web pages to another respected page on the same subject. Their research revolved around B. F. Skinner’s work with pigeons. Skinner’s work, “. . . relies primarily on the superior trainability of the domestic pigeon (Columba Livia) and its unique capacity to recognize objects regardless of spatial orientation.” The trademarked Pigeon Rank is now used by Google to rank pages. They reasoned that if these millions of pigeons (web authors) linked to another site, it was because they thought it was relevant and authoritative. So, by using the Pigeon Rank of the original page that links to another page (called a back link), the second page thus gets its rank modified for how many links that page has. Google does not act as an authority on anything; rather it relies on the (pigeon) expert author to determine what pages are also expert on their area of expertise, and select the best to link to as references or additional expertise. Like a pigeon that can spatially select a seed, from hundreds of pebbles, the author, knowledgeable about a narrow subject selects other expert content pages to link to.
So, the sum of the importance of all pointing (back) links becomes the rank of the target page. The logic works like this: if Don McCue says Dr. Larry Burgess knows something about the Smileys, most informed members would take that as a more authorities lead than if Ron Burgess said the same about McCue.”
Of course Google has hardly stood still in this decade, and has made many alterations and tweaks to the algorithms. While the basic concept of search is similar, today, the webmaster has almost no part to play about how the page is ranked. Google uses the copy alone and the backlinks to set up its scores for each page ranking. And it penalizes pages with what it calls black hat methods of tricking the system. Indeed, we see several websites a year where an uninformed webmaster attempts to increase search results using these black hat tricks, resulting in black hole websites where only the company name will find the website. These circumstances are disastrous for the companies or institutions that hope to attract visitors based on key words and phrases. But we digress.
This system has a few important parts. First, it looks at the copy on the page, based on clear communications, about narrow subjects; it determines what the page is about. General topics or two many topics are confusing, and therefore lower the score. Headlines that are supported by the first couple of sentences, and at least1600 words on the subject are preferred. Second, expert content on a page that sites external “expert” sources give additional “points.” Finally, “authority” depends on the network of inbound links to the page. This may be the most important element.
This works this way. Your Fortnightly paper on the website will be scanned (in as little as 24 hours now), and the topics will be determined. These will include the primary subjects. Google will also index many phrases (perhaps hundreds) and individual words to help in the search results.
In this paper I could have simply left the title Google: Fencing In the Wild West. But while this title may be intriguing to the human, the Google robot cannot tell what it is about. I added “Why new websites will never get top search listings,” so Google would have a better idea about what the paper is really about. The first paragraph, “The days of equal opportunity on Google may be over. New content has new hurtles to getting top search results,” is an attempt at repeating and restating the point of the paper, before going to set the stage.
This may not work for your old creative writing professor, but it is the new reality for getting found online. If Google can’t figure out the subject or theme of the paper, it gets a D-. (On a separate note, this has a profound impact on the existing academic institution, and is largely not understood by academicians. Again a digression.)
But perhaps the most important element beyond writing the copy is who also reads and links to this paper. This is what makes up the “authority” score for each page. Authority is determined by how many other pages with authority, and the same subject, link to your page.
For instance, in my paper of ten years ago, I use the example of Rheumatoid Arthritis as a search result. Today the top result is exactly the same as it was then, Arthritis.org. This is the home page of the website that is titled, “Rheumatoid Arthritis.” The first sentence is “Rheumatoid arthritis (RA) is an autoimmune disease in which your body’s immune system, and etc.”The paragraph continues to describe the condition, and the sub heads down the page are: People, Symptoms, Treatment, and Self-care.
The page itself is doubtless, supported by many, internal links created by the navigation of the website itself. These pages are all about arthritis. Many of these pages have also been linked by other websites for narrow subjects such as, “managing arthritis, fibromyalgia, Gout and Lyme disease.”
Each page gets a score and passes its score on to linked pages for the key phrase Rheumatoid Arthritis. This can occur several levels down as many millions of cross links are catalogued. See figure 1.
Figure 1. Authority links from internal and external sources.
A third level page, with few back-links may have a lower score on the same subject, and links to a second level page, part of its score is added to the next level up, and the same for the second level to the top level page. This complex network allows each page to gain authority based on similar pages. John’s Hopkins University may have expert doctors who have written expert pages on Rheumatoid Arthritis, but the hospital website has by necessity, many other subjects to write about as well. So its one page on Rheumatoid Arthritis, is dwarfed by the Arthritus.org site, on that subject, even if they were to link to the Johns Hopkins site.
So what does all this have to do with fencing in the Internet prairies? In the case of Rheumatoid Arthritis, much is to be gained by having the top website. Most people pick one of the top three listed in the search results. Just a few percent of people make it to the second page of search results. Because of the commercial value that can be created by remedies to Rheumatoid Arthritis, these pages become extremely valuable to pharmaceutical companies, hospitals, doctors, and other allied industries. Today the next websites on a Google search are Mayo Clinic, Wikipedia, WebMD and Medicinenet, a mix between non-profits, and commercial websites.
Some of these sites have extensive content because that is their purpose; others have extensive content because they sell remedies, or have another interest in people who are searching for Rheumatoid Arthritis. In either case, Google ranks the content based on its study of which are the best websites about the subject.
Back to the question of the Internet prairie. With 15 or even twenty years of content development, many websites have tens of thousands of articles. The more content on each subject, the more links to those articles may have developed. Because the links rarely go away, these links continue to accumulate. Further, once a piece of content is written and the author adds links, they tend not to go back and manage or change the content or links.
Now that these relationships are set, when searching for information, we all tend to use the top three website as authority so the chances that you use a top site are much, much higher than a lowly ranked site, so the top pages are perpetuated. They are the digital farmers that have built digital fences.
The prairie is being “fenced in” because for the major and basic terms we use every day, they already have very good information. So the farms with fences, simply maintain their positions. These farms unlike owning land, are not absolute, but those who watch these things monitor their positions and will pour money into new content when important pages slip one position. They many not own this Internet real-estate, but they have very long-term leases on them!
Major institutions and businesses have much to lose on each slip of the search rank. Searchengingwatch.com reported that the top position gets one third of all traffic clicks. The second position is 17.6% and the third 11.4%. The top 5 get 81% of the traffic. How hard is it to get a top ranking? The search term Rheumatoid Arthritis returns over 24,300,000 webpage results. Making the first page results in the probability range of getting hit by lightning several times a year for several years in a row!
So it is clear that being a top performer is critical to gaining natural search traffic. However not all search terms are as crowded as the our example term of Rheumatoid arthritis. So let’s look at a few other terms for comparison.
Insurance 1,180,000,000, top website Geico. How easy will it be for a local business to compete with that? Or a much narrower term, compounding pharmacy? Still hugely competitive at over 8 million.
| Search Term | Competing Pages | Top Website (on 11/15/14) |
| insurance | 1,180,000,000 | Geico |
| compounding pharmacy | 8,130,000 | Wikipedia |
| Men’s Italian Suits | 1,270,000 | italsuit.com |
| college | 933,000 | The college board admissions |
| men’s razors | 744,000 | Wal-Mart, men’s grooming |
| dog cleanup bags | 697,000 | Amazon |
| “solid brass” antique door knobs craftsman style | 47,300 | House of antique hardware |
| Brithinee Electric | 5,400 | Brithinee Electric (and 6 of top 10) |
Even a very specific search such as “solid brass” antique door knobs craftsman style, produces over 47,000 pages. So how would a new company, compete with these established digital farmers? This is the point of the paper. They would compete online with great difficulty, unless the search was local, or the name of the company was part of the search term such as Brithinee Electric.
Google does take special care in helping local businesses get found, however even in the term Dentists (in Redlands) returns pages ad pages of dentists. Local Google search listing plotted on a map lists just 8 of the dozens of dentists in town.
So how much does it cost to get found even in a very narrow geographic area online? The cost varies, and this is because while websites are literally free today if you set up your own, the cost of creating unique, and authorative content can cost thousands in the experts time, or buy hiring professional web writers to do it. Overcoming the top 8 slots for dentists, means adding more content than the existing websites over several years. If larger practices continually add depth of content to their website, then the cost of matching and then overcoming the continual, constant posting is a horse race driven by money, assuming that the talents of the webmaster are equivalent.
Google has always modified its secret algorithms for what is sees as the best approach for Internet search. Today this includes personal preferences. So on top of the main issues mentioned above, each search can be altered slightly for the computer (or cell phones) location, personal search habits and patterns, and individual demographics. This does open up search for smaller websites, based on location, but it also can inject preferences such as websites previously visited, and interests. But these “interests” are mostly created by previous searches you made. Adding a site preference of a low ranking website would probably require looking it up by company or brand name. This is mostly driven by advertising online and offline.
So as you see, the Internet is no longer a wide open range with opportunity for all who enter. As other markets, this one increasingly has been settled and those who understand it are working diligently to maintain the digital fences. The general web searcher will scarcely know the undercurrents of what is happening, because the top websites are in fact very close to the best information available on those subjects. But underneath websites will be battling it out with content.
So what is the future of the new website? Well for those entities with unique names and geographies, traditional branding techniques will create demand based on the brand. But in addition Google is happy to offer advertising, naturally. So websites can attract visitors the old fashioned way. However this is a variable cost paid each time anyone clicks on the advertising link; he more visitors the higher the cost. But even in what seems like it should be a straight pay to play, Google requires even more.
Ads that are not “efficient” in other words don’t perform well in terms of clicks and successful visits (those where the visitors actually stays on the site), the cost will rise, causing the cost per click to go up. Well written ads, and those where the subject is very appropriate for the search term, can get a discount in effect. This incentive is meant to keep advertisers transparent, so that even a click on an ad will in fact be the right answer to the search term quest.
Naturally writing these ads is somewhat of a skill set that small business people don’t necessarily have. So they must learn the complicated world of internet advertising and as well as how to compare it to the costs of increasing natural search costs.
This often looks like a typical rise and run chart, where the need for advertising falls as the natural search becomes more successful. The juncture of the lines becomes the switch of emphasis from one to the other, but the overall cost of the visitor traffic tends to be the same.
The free range of the Internet is clearly dead, when fighting for the valuable traffic that can result in eventual or immediate purchases. The digital farmer, now tends his digital Internet turf just as he does his physical assets to create a return worth the time and effort of being in business.
Meanwhile companies that recognized the commerce value of the Internet early such as Amazon, continue to shorten the distribution pipeline by providing faster delivery, (same day delivery will become common in the near future), and hording the key words of hundreds of thousands of products worldwide.
Some call this the Amazon effect, but contrary to the term, the distribution channel has been shortening since local craftsmen were supplanted by more efficient larger groups of simple assembly lines and later factories.
The ability to compete with the local craftsman was created by cheaper transportation. Canals, were much cheaper than wagon loads overland. Steam power increase the speed, rail beat the water and was more geographically controllable, and today planes and trucks compliment and compete with air and rail.
The distribution systems that were created meshed with transportation. Wholesale warehouses stored inventory which provided more consistent supply, and retailers created convenient locations and end services for the customer.
Conclusion
The Internet is now competing with the retailer in an effective way. It provides many times the variety of an entire city’s worth of retail inventory, and modern logistics anticipate the demand for each item for each region which allows quick delivery. Amazon is not the effect; the Internet combined with quick transportation is the effect. Amazon simply uses both.
The effectiveness of Amazon, Ebay, and other very large online commerce websites, is assisted by the interlinking of internal and external websites. The structure of this interlinking and the Google scheme for search is creating a kind of monopoly for each narrow market niche, namely each search phrase.
Google on the exterior has a simple and noble goal; to provide the most relevant information for each searcher. They also tout a moto: Do no evil. One hopes that Google will continue to modify its search algorithms in some fashion to allow the newcomers to the table too. But currently this writer believes that the trend is modeling the physical economy, the digital farm costs money to maintain and produce too.