While users are “real” people, user agents do not refer to people, but to programs. They carry out automatic data queries from the server, for example from search engines, and thus form the interface between the user and the network service. The user agent is transmitted via the HTTP header.
User agent types
According to theinternetfaqs.com, user agents can be divided into the following types:
Browser: e.g. Firefox, Internet Explorer, Chrome. Browsers are required for the uniform display of the website.
Search engine crawlers: e.g. Googlebot, MSN-Bot. You visit websites regularly in search of new information and changes to websites.
Web applications: e.g. Flash Player, Adobe Acrobat Reader. This type includes all programs that are required to execute a file and that send requests to the server for this purpose. They are required, for example, to play videos and music or to open text / graphic files.
Purpose of the user agent
Each user agent has its own identifier as an identification. This identifier is also known as the agent name. It is transmitted via its HTTP header. With this information, the web server can deliver other information to the user, which is important, for example, for browser users with a mobile device. Because the origin information shows the user a page optimized for the device.
In addition, the user agent can be used to distinguish people from machines, i.e. search engine crawlers. Your request is very similar to human requests, except that information such as “Googlebot / 2.1” or “msnbot-media / 1.0” is stored here. This distinction is especially important for statistical analysis.
How does the transfer of the user agent work?
The transmission of the user agent via the header takes place every time a page is called up. The header contains the following information:
– Name and version of the calling program
– Name of the operating system and its version
– Possibly plugins for browsers
It can look like this:
Mozilla / 5.0 (Windows; U; Windows NT 5.1; de; rv: 1.8.1.12) Gecko / 20080201 Firefox / 2.0.0.12
When a browser, application or crawler accesses a document, the user agent is automatically notified. The value is not standardized. Therefore, there are different information both in terms of content and structure. From a technical point of view, a user agent can send any information. That is why he sometimes transmits false information in order to conceal his identity and to pretend to be a different program.
User agent – importance for search engine optimization
If the requesting program receives content tailored to its user agent, this is referred to as agent name delivery. Agent Name Delivery is used in search engine optimization by delivering different information to real users than search engine crawlers. This practice is known as cloaking. The aim is to provide the real user with lavishly designed pages via their browser view and simply “present” a simple structure to the crawler.
This search engine technology falls under the category of spam and violates the Google Webmaster Guidelines. Since the manipulation of the user agent is quite easy to do, this spam method hardly works anymore. Most search engines now output their crawlers with a user agent ID that is equivalent to a normal browser.
