- General Guidelines Overview
- Introduction to Search Quality Rating
- Part 1: Page Quality Rating Guideline
- Part 2: Understanding Mobile User Needs
- Part 3: Needs Met Rating Guideline
- Part 4: Definition of Child Pornography
General Guidelines Overview
As a search quality rater, you would be working on different evaluation projects, but the guidelines provided in this search quality brief would be applicable in many different scenarios.
Introduction to Search Quality Rating
The Search Experience
0.1 The Purpose of Search Quality Rating
0.2 Raters Must Represent People in their Rating Locale
0.3 Browser Requirements
0.4 Ad Blocking Extensions
0.5 Internet Safety Information
0.6 The Role of Examples in these Guidelines
0.0 The Search Experience
With a plethora of information available on the World Wide Web, individuals search the Internet with different motives and different purposes that could range from looking for adorable, cute pictures of puppies to finding information regarding certain medical conditions.
Therefore, it must be ensured that the search results being returned to an individual’s query should be unoffensive, not contain explicit material unless specifically asked for, and should deliver the required information from a trustworthy source.
0.1 The Purpose of Search Quality Rating
Your role as a search quality evaluator will not impact the ranking of any website or search result rather it is focused on determining the quality of a search engine in the terms of the results it produces and how relevant and effective it is in delivering what a user wants from it in relevance to their demographics. To further enhance the experience you must know how to use operators and other advanced search options.
0.2 Raters Must Represent People in their Rating Locale
You must be aware of the task language of the people of your locale, in case of the contrary please contact your employer. Ratings should be unbiased, not represent your personal beliefs, and should be according to the instructions present in this manual.
0.3 Browser Requirements
Check with your company about browser requirements and allowed extensions/addons.
0.4 Ad Blocking Extensions
Do not use adblocking extensions unless clearly instructed to do so.
0.5 Internet Safety Information
Do not visit websites that ask you to download certain files. Always have an antivirus and anti-spyware installed and active on your system.
0.6 The Role of Examples in these Guidelines
The examples used in these guidelines were the latest at the time of its development.
Part 1: Page Quality Rating Guideline
- 1.0 Introduction to Page Quality Ratin
- 2.0 Understanding Webpages and Websites
- 2.1 Important Definitions
- 2.2 What is the Purpose of a Webpage?
- 2.3 Your Money or Your Life (YMYL) Pages
- 2.4 Understanding Webpage Content
- 2.5 Understanding the Website
- 2.6 Reputation of the Website and Creator of the Main Content
- 3.0 Overall Page Quality Rating
- 3.1 Page Quality Rating: Most Important Factors
- 4.1 Characteristics of High Quality Pages
- 4.2 A Satisfying Amount of High Quality Main Content
- 5.0 Highest Quality Pages
- 6.0 Low Quality Pages
- 7.0 Lowest Quality Pages
- 8.0 Medium Quality Pages
- 9.0 Page Quality Rating Tasks
- 9.1 Instructions for Rating Page Quality Tasks
- 9.2 Reputation and E-A-T (expertise, authoritativeness and trustworthiness): Website or the Creators of the Main Content?
- 10.0 Page Quality Criteria for Specific Types of Pages
- 10.1 Ratings for Encyclopedia Pages
- 10.2 Ratings for Pages with Error Messages or No MC
- 10.3 Ratings for Forums and Q&A pages
- 11.0 Page Quality Rating FAQs
1.0 Introduction to Page Quality Rating
A Page Quality (PQ) rating task consists of a URL and a grid to record your observations, to guide your search. The main task is to analyze whether the page achieves its objective or not.
2.0 Understanding Webpages and Websites
PQ rating requires an in-depth understanding of websites. We’ll start with the basics. These are a must-read.
2.1 Important Definitions
- A search engine is a tool to help people in interacting with content available on the internet.
- The term user refers to someone who’s trying to find some information on the Internet.
- A web page is connected to the World Wide Web and can be visited using a web browser.
- A URL is the address of a web page. A website is a group of web pages . A home page of a website is the main web page of the site.
- A subpage is any page on a website other than the home page. A webmaster is someone responsible for maintaining a website .
2.2 What is the Purpose of a Webpage?
Before you begin page quality rating you must ensure that you truly understand the purpose of a webpage the purpose can start anywhere from earning money or just disseminating news or information. First off you must do so because then you can only rate a page on its quality and given how effective it is if you know its objective or purpose.
2.3 Your Money or Your Life (YMYL) Pages
Some pages can impact a person’s financial status, safety, well-being, or health. Such pages are known as “Your Money or Your Life” pages. Therefore, the rating standards for these pages are of high quality otherwise they can impact a person’s life negatively. Some examples of such pages are government pages, websites of financial institutions, or verified medical websites.
2.4 Understanding Web-Page Content
Content on any web page is can be categorized into three main types. You need to understand and differentiate between these based on your best judgment.
Main Content (MC) is any part of the page that directly helps the page achieve its purpose.
Supplementary Content (SC) is also important. SC can help a page better achieve its purpose or it can detract from the overall experience.
Many pages have advertisements/monetization (Ads). Without advertising and monetization, some web pages could not exist because it costs money to maintain a website.
2.5 Understanding the Website
To know more about a website and develop an understanding look at the web pages in a connection rather than separate entities. Websites are very usually very eager to talk about themselves. There are three facets to this.
Finding the Homepage
Examine the landing page of the URL in your PQ rating task or Find and click on the link labeled “home” or “main page” or try using “Ctrl-F”(command-F on a Mac) to search the page for the text “home”.
Finding Who is responsible for the Website
Finding About Us Information
2.6 Reputation of the Website and Creator of the Main Content
An important part of PQ rating is understanding the reputation of the website. If the creator of the MC is different from the creator of the website, it’s important to understand the reputation of the creator as well. Your job is to thoroughly gauge the website repute as well as the credibility of the creator of MC through research, customer reviews, and trustworthy review websites.
3.0 Overall Page Quality Rating
At a high level, here are the steps of Page Quality rating:
- Assess the t rue purpose of the page.
- Assess the potential of the page to be harmful, untrustworthy, or spammy as defined ahead in Section 7 of these guidelines.
- Otherwise, the PQ rating is based on how well the page achieves its purpose using the criteria outlined in the following sections on Lowest, Low, Medium, High, and Highest quality pages.
- On Page Quality rating t asks, you will use the Page Quality sliding scale (slider) to assign the overall PQ rating. The slider looks like this:
3.1 Page Quality Rating:
Most Important Factors
Here are the most important factors to consider when selecting an overall Page Quality rating:
- The Purpose of the Page
- Expertise, Authoritativeness, Trustworthiness: This is an important quality characteristic. Use your research on the additional factors below to inform your rating.
- Main Content Quality and Amount: The rating should be based on the landing page of the task URL.
- Website Information/information about who is responsible for the MC: Find information about the website as well as the creator of the MC.
- Website Reputation/reputation about who is responsible for the MC: Links to help with reputation research will be provided.
4.1 Characteristics of High Quality Pages
High-quality pages exist for almost any beneficial purpose, from giving information to making people laugh to expressing oneself artistically to purchasing products or services online. What makes a High-quality page? A High-quality page should have a beneficial purpose and achieve that purpose well.
4.2 A Satisfying Amount of High Quality Main Content
For each page, you evaluate, spend a few minutes examining the MC before concluding it. Read the article, watch the video, examine the pictures, use the calculator, play the online game, etc. Remember that MC also includes page features and functionality, so test the page out. For example, if the page is a product page on a store website, but at least one product is in the cart to make sure the shopping cart is functioning. If the page is an online game, spend a few minutes playing it.
5.0 Highest Quality Pages
The highest quality pages are created to serve a beneficial purpose and achieve their purpose very well. The distinction between High and Highest is based on the quality and quantity of MC, as well as the level of reputation.
6.0 Low Quality Pages
Low-quality pages may have been intended to serve a beneficial purpose. However, Low-quality pages do not achieve their purpose well because they are lacking in an important dimension, such as having an unsatisfying amount of MC, or because the creator of the MC lacks expertise for the page. If a page has one or more of the following characteristics, the Low rating applies:
- An inadequate level of Expertise, Authoritativeness, and Trustworthiness (E-A-T).
- The quality of the MC is l ow.
- There is an unsatisfying amount of MC f or the purpose of the page.
- The title of the MC is exaggerated or shocking.
- The Ads or SC distracts from the MC.
- There is an unsatisfying amount of website information or information about the creator of the MC for the purpose of the page (no good reason f or anonymity).
- A mildly negative reputation f or a website or creator of the MC, based on extensive reputation research. If a page has multiple Low quality attributes, a rating lower than Low may be appropriate.
7.0 Lowest Quality Pages
Following are some types of lowest quality pages
- Harmful to self or other individuals
- Harmful to specified groups
- Harmfully misleading information
- Untrustworthy pages
- Spammy Webpages
8.0 Medium Quality Pages
There are two types of Medium quality pages:
- Nothing wrong, but nothing special – The page achieves its purpose; however, it does not merit a High quality rating, nor is there anything to indicate that a Low quality rating is appropriate.
- Mixed with strong High quality rating characteristics – The page or website has strong High quality rating characteristics, but also has mild Low quality characteristics. The strong High quality aspects make i t difficult to rate the page Low.
9.0 Page Quality Rating Tasks
Page quality rating tasks might seem difficult at first but with this guideline, in place, we recommend that you read it a couple of times and refer to sections so that you can develop a total understanding of what is required from a search quality evaluator. But do understand that these guidelines do not cover each aspect of every page out there, you’re required to develop your acumen and you should rate a page high or low quality depending on your understanding as well as the requirements of your locale.
9.1 Instructions for Rating Page Quality Tasks
Page quality tasks are usually broken down into four major steps. But these are not limiting and can be reduced to increase depending on the requirement of the task, regional requirements, and specific business requirements.
- Some initial questions about the task landing page.
- A “PQ grid” to record your observations about PQ characteristics of the landing page.
- The Overall PQ rating slider which records your Overall PQ rating.
- A comment box to explain your rating.
9.2 Reputation and E-A-T (expertise, authoritativeness and trustworthiness):
Website or the Creators of the Main Content?
You must evaluate the expertise, authoritativeness, and trustworthiness of a website as well as the credibility and repute of the creators of its MC(Main Content). This is especially important when different pages on the website have different creators.
10.0 Page Quality Criteria for Specific Types of Pages
10.1 Ratings for Encyclopedia Pages
We may not always know the author of the specific encyclopedia article, and therefore must rely on website reputation research to determine the E-A-T of the article. High and Highest quality ratings should only be used for encyclopedias with very good reputations for accuracy and expertise, where the article itself is well-researched with appropriate references cited.
10.2 Ratings for Pages with Error Messages or No MC
Some pages are temporarily broken pages on otherwise functioning websites, while some pages have an explicit error (or custom 404) message. In some cases, pages are missing MC as well. Please think about whether the page offers help for users—did the webmaster spend time, effort, and care on the page?
10.3 Ratings for Forums and Q&A pages
For this category keep the following points in mind :
- The Main Content on forum and Q&A pages includes both the question as well as the answers/responses and resulting discussions.
- Rate forum and Q&A pages from the point of view of a user who visits the page, rather than a participant involved in the discussion.
11.0 Page Quality Rating FAQs
Part 2: Understanding Mobile User Needs
- 12.0 Understanding Mobile Users, Mobile Queries, and Mobile Results
- 12.1 Important Rating Definitions and Ideas
- 12.2 Understanding the Query
- 12.3 Locale and User Location
- 12.4 Queries with an Explicit Location
- 12.5 Queries with Multiple Meanings
- 12.6 Query Meanings Can Change Over Time
- 12.7 Understanding User Intent
- 12.8 Understanding Result Block
- 12.9 Rating on Your Phone Issues
12.0 Understanding Mobile Users, Mobile Queries, and Mobile Results
People rely on smartphones to do a lot for themselves nowadays. But at the same time we face some challenges that we don’t face while using a desktop computer or laptop:
- Entering data may be cumbersome: typing is difficult on mobile smartphones, and when users speak to their phones instead of typing, voice recognition may not always be accurate.
- Small screen sizes make it difficult to use some phone features, apps, and webpages.
- Some webpages are difficult to use on a mobile phone. Website navigation can be difficult as menus and navigation links may be small. Webpages may require left-to-right scrolling to read text. Images may not fit on the screen. In addition, many mobile devices cannot access webpages with Flash or other similar features.
- Internet connectivity can be slow and inconsistent for mobile users going in and out of networks. App opening, recognition of voice commands, and webpage load times can be very slow on a mobile phone.
12.1 Important Rating Definitions and Ideas
- Query: This refers to the word(s) and/or number(s) that a user types or speaks into a mobile phone.
- User: The user is the person trying to find information or accomplish a task by typing or speaking into a mobile phone with a small screen (i.e., size of a smartphone, not a tablet).
- User Intent: When a user types or speaks a query, he or she is trying to accomplish something. We refer to this goal as the user intent.
- Locale: All queries have a locale, which is the language and location f or the task. Locales are represented by a t wo-letter country code.
- User Location: This tells us where the user is located and should be inferred from the map provided.
- Search Engine Results Page (SERP): The page a search engine shows after a user enters a query in the search box.
- Result: We will use the word result to refer to the result block and the landing page.
- Device Actions: Mobile phones and other devices can respond to voice commands to perform many actions, such as setting an alarm or opening an app.
12.2 Understanding the Query
Understanding the query is the first step in evaluating the task. Remember, a query is what a user types or speaks into a mobile phone.
12.3 Locale and User Location
All queries have a task language and task location (referred to in rating tasks as the “Locale”). The locale is important for understanding the query and user intent. Users in different locations may have different expectations for the same query.
12.4 Queries with an Explicit Location
Sometimes users tell search engines exactly what kinds of results they are looking for by adding the desired location in the query, regardless of their user location. We’ll call this location inside the query the “explicit location.” The explicit location makes queries much easier to understand and interpret.
12.5 Queries with Multiple Meanings
Many queries have more than one meaning. For example, the query [ apple] might refer to the computer brand or the fruit.
We will call these possible meanings query interpretations.
- Dominant Interpretation: The dominant interpretation of a query is what most users mean when they type the query. Not all queries have a dominant interpretation. The dominant interpretation should be clear to you, especially after doing a little web research.
- Common Interpretation: A common interpretation of a query is what many or some users mean when they type a query. A query can have multiple common interpretations.
- Minor Interpretations: Sometimes you will find fewer common interpretations. These are interpretations that few users have in mind. We will call these minor interpretations .
12.6 Query Meanings Can Change Over Time
Remember to think about the query and its current meaning as you are rating. We will assume users are looking for current information about a topic, the most recent product model, the most recent occurrence of a recurring event, etc. unless otherwise specified by the query.
12.7 Understanding User Intent
It can be helpful to think of queries as having one or more of the following intents.
- Know query, some of which are Know Simple queries
- Do query, some of which are Device Action queries
- Website query when the user is looking for a specific website or webpage
- Visit-in-person query, some of which are looking f or a specific business or organization, some of which are looking f or a category of businesses
12.8 Understanding Result Block
There are different types of Result Block for every query. To interpret them individually based on requirements.
12.9 Rating on Your Phone Issues
We understand that raters using different phones, operating systems, and browsers may have different experiences. In general, do what you would do naturally, and rate based on your experience.
Part 3: Needs Met Rating Guideline
- 13.0 Rating Using the Needs Met Scale
- 13.1 Rating Result Blocks: Block Content and Landing Pages
- 13.2 Fully Meets (FullyM)
- 13.2.1 Examples of Fully Meets (FullyM) Result Block
- 13.2.2 Examples of Queries that Cannot Have Fully Meets Results
- 13.3 Highly Meets
- 13.3.1 Examples of Highly Meets (HM) Result Blocks
- 13.4 Moderately Meets (MM)
- 13.4.1 Examples of Moderately Meets (MM) Results Block
- 13.5 Slightly Meets (SM)
- 13.5.1 Examples of Slightly Meets (SM) Result Blocks
- 13.6 Fails to Meet (FailsM)
- 13.6.1 Examples of Fails to Meet (FailsM) Result Blocks
- 14.0 The Relationship between Page Quality and Needs Met
- 15.0 Rating Porn, Foreign language, and Did Not Load Results
- 15.1 Porn Flag
- 15.2 Needs Met Rating For Porn Results
- 15.2.1 Needs Met Rating for Clear Non-Porn Intent Queries
- 15.2.2 Needs Met Rating for Possible Porn Intent Queries
- 15.2.3 Needs Met Rating for Clear Porn Intent Queries
- 15.3 Reporting Illegal Images
13.0 Rating Using the Needs Met Scale
- FullyM: Fully meets almost all users’ requirements exclusively.
- HM: Highly meets most users’ requirements.
- MM: Moderately meets means it’s either helpful for many or very helpful for some.
- SM: Slightly meets means many or most users would like to see additional results.
- FailsM: Fails to meet any user requirements.
Rating may be assigned in between any of the two labels as well.
13.1 Rating Result Blocks: Block Content and Landing Pages
Each result needs to be rated for Needs Met Rating. The part that you rate would depend upon both the query and result block.
Special Content Result Block: This should play the largest role in your rating. Some SCRBs may have links to landing pages.
If most users would not click then the rating should be based on block content alone.
If users may click on it then the helpfulness of landing pages is also considered in rating.
Web Search Result Block: Needs a click and landing pages should be evaluated.
Device Action Result Block: Rated on the helpfulness of the action.
For example, What does Love mean
Rating based on the content inside the block as there are no obvious landing pages.
Web Search Result Block:
Rating based on the content of the landing page as the user would need to click on it.
Some more examples where rating should be based on the content inside the block are as under.
Note: This section applies to Needs Met ratings. For SCRBs that have landing pages you may be asked to provide page quality ratings and for that refer to section 14.0.
13.2 Fully Meets (FullyM)
It’s a special rating category used when
- The query and user need should be completely unambiguous.
- Mobile users should be satisfied and require minimum effort for the query’s answer.
- All or almost all users would be completely satisfied.
FullyM should be reserved for complete and perfect responses. Some scenarios where FullyM is appropriate are.
- When users find specific webpage they were looking for.
- When result block completes a device action required by the user.
- When user finds a specific fact that they were looking for. Accuracy of the fact should be checked before rating FullyM.
It may apply in other situations as well. Go for a lower rating when in doubt.
Note: If the result block is almost FullyM but may need additional info HM rating is more appropriate.
Sometimes the specificity of the query may also need to be considered.
13.2.1 Examples of Fully Meets (FullyM) Result Block
13.2.2 Examples of Queries that Cannot Have Fully Meets Results
13.3 Highly Meets
It meets the requirements of most users and results are a good fit for the query. Information pages like encyclopedias must be highly credible to fall in this category. Scientific information pages must conform to the consensus.
A single query may have many HM results. It is especially important for queries that have multiple on-topic results.
13.3.1 Examples of Highly Meets (HM) Result Blocks
13.4 Moderately Meets (MM)
These are results that are satisfying for many users or highly satisfying for some users. Results should still fit the query but they have fewer qualities than HM.
These results are not inaccurate and are generally average to good.
13.4.1 Examples of Moderately Meets (MM) Results Block
13.5 Slightly Meets (SM)
Results that are helpful for a few users. It may have some minor inaccuracies or be outdated. Results may also be too specific or too broad for a higher rating.
Note: Mobiles have higher costs per click and the clicks depend upon titles. Exaggerated or, is leading titles may also be SM or below.
13.5.1 Examples of Slightly Meets (SM) Result Blocks
13.6 Fails to Meet (FailsM)
These results are helpful to either very few or no users at all. Results may be unrelated to the query or incorrect. It fails to meet user requirements or may be completely outdated information.
Results should never be offensive or unpleasant.
- Harmful to Self or Other Individuals (Section 7.1)
- Harmful to Specified Groups (Section 7.2)
- Harmful Misleading Information (Section 7.3)
- Untrustworthy (Section 7.4)
- Spammy (Section 7.5)
- Porn (Section 15.1)
13.6.1 Examples of Fails to Meet (FailsM) Result Blocks
14.0 The Relationship between Page Quality and Needs Met
Needs Met Rating depends upon both query and results
Page Quality rating does not depend on the query
Page quality reading only needs to be given when there is a slider and only depends upon the quality of the content.
Some Guidelines are as under
- Useless results should be rated FailsM even if page quality is good.
- Lower content quality should mean lower Needs Met Rating as well.
- HM rating should be given to helpful, high quality pages. Medium quality pages may be rated HM if other qualities are there.
- Low page quality does not merit an HM rating.
- Low quality on-topic pages should be SM. If the content is completely gibberish it should be FailsM. The only exception is clear website intent in the query where search result should be rated FullyM regardless.
- If the page serves no purpose it should always be rated Lowest page quality.
15.0 Rating Porn, Foreign language, and Did Not Load Results
Assignment of flags does not depend on the query. Screenshot of flags
15.1 Porn Flag
Porn needs to be flagged as porn even if the query is specifically asking for porn.
15.2 Needs Met Rating For Porn Results
15.2.1 Needs Met Rating for Clear Non-Porn Intent Queries
FailsM will be assigned if a query is non-porn related.
15.2.2 Needs Met Rating for Possible Porn Intent Queries
Some queries may have both porn and non-porn intended meanings. They will be rated as though the query is non-porn-related.
15.2.3 Needs Met Rating for Clear Porn Intent Queries
Queries that have clear porn intents should be rated as per relevance to the query. But the page is still to be flagged as porn
15.3 Reporting Illegal Images
Note: Federal laws of task location need to be followed.
Definition of Child Pornography
- 15.4 Foreign Language Flag
- 15.4.1 Using the Foreign Language Flag
- 15.4.2 Needs Met Rating for Foreign Language Results
- 15.5 Did Not Load Flag
- 15.5.1 Using the Did Not Load Flag
- 15.2.2 Needs Met Rating and the Did Not Load Flag
- 15.6 Additional Flags in Some Rating Tasks
- 16.0 Rating Queries with Multiple Interpretations and Intents
- 16.1 Rating Queries with Both Website and Visit-in-Person Intent
- 17.0 Specificity of Queries and Landing Pages
- 18.0 Needs Met Rating and Freshness
- 19.0 Misspelled and Mistyped Queries and Results
- 19.1 Misspelled and Mistyped Queries
- 19.2 Name Queries
- 20.0 Non-Fully Meets Results for URL Queries
- 21.0 Product Queries: Importance of Browsing and Researching
- 22.0 Rating Visit-in-Person Intent Queries
- 22.1 Examples Where User Location Does (and Does Not) Matter
- 23.0 Rating English Language Results in Non-English Locales
- 23.1 Examples of English (and Non-English) Results in Non-English Locales
- 24.0 Rating Dictionary and Encyclopedia Results for Different Queries
If an image depicts someone minor or appears to be a minor in sexually explicit conduct it is called child pornography. It may contain actual minor or computer-generated imagery. Even drawings, cartoons, anime, or sculptures depicting sexually explicit conduct for minors are child pornography.
Even if the pornographic images depict children in a literary, political, artistic, or scientific context, please forward them to your employer. URLs are required to be forwarded as instructed.
Depiction of minors’ genitals does not require them to be uncovered. They can be considered Child pornography even if covered. A person pretending to be a minor in sexually explicit content is not child Pornography but if you cannot tell whether the person being shown in such a scene is over the legal age then it is considered Child Pornography.
15.4 Foreign Language Flag
15.4.1 Using the Foreign Language Flag
If the language of the result is not the task language or the language used by the majority in the task location it is to be flagged as Foreign.
- Assign the flag even if you understand the foreign language.
- Flag should be assigned based on the content of the landing page.
- Flag even if a foreign language page is targeted by the query.
- When In doubt don’t use the foreign language flag.
15.4.2 Needs Met Rating for Foreign Language Results
Most Foreign Language pages would be rated FailsM as people in the task location wouldn’t understand it, unless a query specifically asks for a foreign page, then assign FullyM.
Note: If you can’t read the foreign language content page you don’t have to assign a page quality rating.
15.5 Did Not Load Flag
15.5.1 Using the Did Not Load Flag
The flag is used when an error message load or the landing page is completely blank. The flag is assigned on the loading page, not the result block.
A flag is not assigned for:
- Malware warnings.
- Pages that were removed or expired.
- Inaccessible due to required subscription.
15.2.2 Needs Met Rating and the Did Not Load Flag
Did not Load pages are also assigned FailsM as they are useless. If the page partially loads, the Needs Met Rating is assigned based on the available content. Page Quality Rating is not assigned if a page cannot be evaluated.
15.6 Additional Flags in Some Rating Tasks
Some rating tasks may ask you to rate something that might be upsetting or offensive. Such rating is assigned based on the typical user of your locale.
16.0 Rating Queries with Multiple Interpretations and Intents
Some queries may have multiple interpretations. Conform to the following guidelines while dealing with them:
- Very helpful result for dominant interpretation is HM some might even be FullyM.
- Very helpful result for a common interpretation is HM or MM.
- Very helpful result for a Minor interpretation is SM or lower.
- No chance interpretations are FailsM.
16.1 Rating Queries with Both Website and Visit-in-Person Intent
Users may have dual intent of either visiting a website or visiting some store in-person. If the intent is not clear then results satisfying only one of the intents is never rated Fully Meets.
17.0 Specificity of Queries and Landing Pages
Queries may be specific or general as shown in the table, but the rating should always depend upon how helpful the result is. If the query is a broad category then the most popular results according to locale should be very helpful.
18.0 Needs Met Rating and Freshness
Some queries require very recent information.
Some queries demand breaking news. This is when old pages will be rated FailsM. Some queries may be looking for recent or timeless information. Page Quality rating is generally not based on freshness unless the page has been abandoned.
Note: Websites show different dates depending on their settings, if you want to know the date of the content, try the “Wayback Machine” on the Internet Archive.
19.0 Misspelled and Mistyped Queries and Results
19.1 Misspelled and Mistyped Queries
If queries are obviously misspelled, rate the results on the user intent. For queries that are not very obvious in misspelling, rate the results as per the user spellings.
19.2 Name Queries
Follow the given examples
20.0 Non-Fully Meets Results for URL Queries
URL queries may be issued by some users to find info about the website so the rating for any result that is not the website itself may not be FailsM. Usage statistics, however, are not what most users are looking for.
21.0 Product Queries: Importance of Browsing and Researching
User queries may have the intent to buy or seek information and they may be unspecified as well. Users also enjoy online browsing and need authoritative information on the products while scouting them online. You have to keep in mind all these factors while assigning Needs Met Rating to results. Results for product queries may be important for both your money and your life (YMYL.)
22.0 Rating Visit-in-Person Intent Queries
Users can travel different range of distances while looking for different services. For example, they may be willing to travel farther for a clinic than for a famous coffee shop. So you have to use your judgment on the meaning of “nearby”.
22.1 Examples Where User Location Does (and Does Not) Matter
23.0 Rating English Language Results in Non-English Locales
Even if you can understand English, the Needs Met Rating is supposed to be based on the users in your Locale. Unless a query specifically asks for results in English, for a non-English locale, results in English should be considered useless.
Important: There may be many languages to consider including task language, official language, local dialects, etc. When you face doubt, prefer the results in task language unless the query specifically indicates otherwise.
23.1 Examples of English (and Non-English) Results in Non-English Locales
Here are some examples that include proper nouns in non-English queries.
Here are some examples where users would still prefer results in the local language.
There may be queries where English language results would satisfy the user more even if the query is in task language. You will have to use your judgment for assigning ratings.
In locales where English is the official language, results in English would be preferred.
24.0 Rating Dictionary and Encyclopedia Results for Different Queries
You have to judge whether a user is looking for the meaning of a word or the word is commonly known and intent may be different from Dictionary results. You should mostly reserve Dictionary results for queries that include “What is” or “What is the meaning of.”
Important: If a commonly known word is used in the query, an SM rating may be appropriate for a dictionary result.