Search Quality Evaluators Guidelines – Brief

  Published on February 21 |   28 minutes read
bg

General Guidelines Overview

As a search quality rater, you would be working on different evaluation projects, but the guidelines provided in this search quality brief would be applicable in many different scenarios.

Introduction to Search Quality Rating

The Search Experience
0.1 The Purpose of Search Quality Rating
0.2 Raters Must Represent People in their Rating Locale
0.3 Browser Requirements
0.4 Ad Blocking Extensions
0.5 Internet Safety Information
0.6 The Role of Examples in these Guidelines

 0.0 The Search Experience

With a plethora of information available on the World Wide Web, individuals search the Internet with different motives and different purposes that could range from looking for adorable, cute pictures of puppies to finding information regarding certain medical conditions.

Therefore, it must be ensured that the search results being returned to an individual’s query should be unoffensive, not contain explicit material unless specifically asked for, and should deliver the required information from a trustworthy source.

0.1 The Purpose of Search Quality Rating

Your role as a search quality evaluator will not impact the ranking of any website or search result rather it is focused on determining the quality of a search engine in the terms of the results it produces and how relevant and effective it is in delivering what a user wants from it in relevance to their demographics. To further enhance the experience you must know how to use operators and other advanced search options.

0.2 Raters Must Represent People in their Rating Locale

You must be aware of the task language of the people of your locale, in case of the contrary please contact your employer. Ratings should be unbiased, not represent your personal beliefs, and should be according to the instructions present in this manual.

0.3 Browser Requirements

Check with your company about browser requirements and allowed extensions/addons.

0.4 Ad Blocking Extensions

Do not use adblocking extensions unless clearly instructed to do so.

0.5 Internet Safety Information

Do not visit websites that ask you to download certain files. Always have an antivirus and anti-spyware installed and active on your system.

0.6 The Role of Examples in these Guidelines

The examples used in these guidelines were the latest at the time of its development.

Part 1: Page Quality Rating Guideline

1.0 Introduction to Page Quality Rating

A Page Quality (PQ) rating task consists of a URL and a grid to record your observations, to guide your search. The main task is to analyze whether the page achieves its objective or not.

2.0 Understanding Webpages and Websites

PQ rating requires an in-depth understanding of websites. We’ll start with the basics. These are a must-read.

2.1 Important Definitions

  1. A search engine  is a tool to help people  in interacting with content available on the internet.
  2. The term user refers to someone who’s trying to find some information on the Internet.
  3. A web page is connected to the World Wide Web and can be visited using a web browser.
  4. A URL is the address of a web page. A website is a group of web pages . A home page of a website is the main web page of the site. 
  5. A subpage is any page on a website other than the home page. A webmaster is someone responsible for maintaining a website .

2.2 What is the Purpose of a Webpage?

Before you begin page quality rating you must ensure that you truly understand the purpose of a webpage the purpose can start anywhere from earning money or just disseminating news or information. First off you must do so because then you can only rate a page on its quality and given how effective it is if you know its objective or purpose.

2.3 Your Money or Your Life (YMYL) Pages

Some pages can impact a person’s financial status, safety, well-being, or health. Such pages are known as “Your Money or Your Life” pages. Therefore, the rating standards for these pages are of high quality otherwise they can impact a person’s life negatively. Some examples of such pages are government pages, websites of financial institutions, or verified medical websites.

2.4 Understanding Web-Page Content

Content on any web page is can be categorized into three main types. You need to understand and differentiate between these based on your best judgment.

 Main Content (MC) is any part of the page that directly helps the page achieve its purpose.

Supplementary Content (SC) is also important. SC can help a page better achieve its purpose or it can detract from the overall experience.

Many pages have advertisements/monetization (Ads). Without advertising and monetization, some web pages could not exist because it costs money to maintain a website.

2.5 Understanding the Website

To know more about a website and develop an understanding look at the web pages in a connection rather than separate entities. Websites are very usually very eager to talk about themselves. There are three facets to this.

Finding the Homepage

Examine the landing page of the URL in your PQ rating task or Find and click on the link labeled “home” or “main page” or try using “Ctrl-F”(command-F on a Mac)  to search the page for the text “home”.

Finding Who is responsible for the Website

Finding About Us Information

2.6 Reputation of the Website and Creator of the Main Content

An important part of PQ rating is understanding the reputation of the website. If the creator of the MC is different from the creator of the website, it’s important to understand the reputation of the creator as well. Your job is to thoroughly gauge the website repute as well as the credibility of the creator of MC through research, customer reviews, and trustworthy review websites.

3.0 Overall Page Quality Rating

At a high level, here are the steps of Page Quality rating:

  1. Assess the t rue purpose of the page.
  2. Assess the potential of the page to be harmful, untrustworthy, or spammy as defined ahead in Section 7 of these guidelines.
  3. Otherwise, the PQ rating is based on how well the page achieves its purpose using the criteria outlined in the following sections on Lowest, Low, Medium, High, and Highest quality pages.
  4. On Page Quality rating t asks, you will use the Page Quality sliding scale (slider) to assign  the overall PQ rating. The slider looks like this:
page-quality-rating-meter

3.1 Page Quality Rating:

Most Important Factors

Here are the most important factors to consider when selecting an overall Page Quality rating:

  1. The Purpose of the Page
  2. Expertise, Authoritativeness, Trustworthiness: This  is an important quality characteristic. Use your research on the additional factors below to inform your rating.
  3. Main Content Quality and Amount: The rating should be based on the landing page of the task URL.
  4. Website Information/information about who is responsible for the MC: Find information about the website as well as the creator of the MC.
  5. Website Reputation/reputation about who is responsible for the MC: Links to help with reputation research will be provided.

4.1 Characteristics of High Quality Pages

High-quality pages exist for almost any beneficial purpose, from giving information to making people laugh to expressing oneself artistically to purchasing products or services online. What makes a High-quality page? A High-quality page should have a beneficial purpose and achieve that purpose well.

4.2 A Satisfying Amount of High Quality Main Content

For each page, you evaluate, spend a few minutes examining the MC before concluding it. Read the article, watch the video, examine the pictures, use the calculator, play the online game, etc. Remember that MC also includes page features and functionality, so test the page out. For example, if the page is a product page on a store website, but at least one product is in the cart to make sure the shopping cart is functioning. If the page is an online game, spend a few minutes playing it.

5.0 Highest Quality Pages

The highest quality pages are created to serve a beneficial purpose and achieve their purpose very well. The distinction between High and Highest is based on the quality and quantity of MC, as well as the level of reputation.

6.0 Low-Quality Pages

Low-quality pages may have been intended to serve a beneficial purpose. However, Low-quality pages do not achieve their purpose well because they are lacking in an important dimension, such as having an unsatisfying amount of MC, or because the creator of the MC lacks expertise for the page. If a page has one or more of the following characteristics, the Low rating applies:

  1. An inadequate level of Expertise, Authoritativeness, and Trustworthiness (E-A-T).
  2. The quality of the MC is l ow.
  3. There is an unsatisfying amount of MC f or the purpose of the page.
  4. The title of the MC is exaggerated or shocking.
  5. The Ads or SC distracts from the MC.
  6. There is an unsatisfying amount of website information or information about the creator of the MC for the purpose of the page (no good reason f or anonymity).
  7. A mildly negative reputation f or a website or creator of the MC, based on extensive reputation research. If a page has multiple Low quality attributes, a rating lower than Low may be appropriate.

7.0 Lowest Quality Pages

Following are some types of lowest quality pages

  1. Harmful to self or other individuals
  2. Harmful to specified groups
  3. Harmfully misleading information
  4. Untrustworthy pages
  5. Spammy Webpages

8.0 Medium Quality Pages

There are two types of Medium quality pages:

  1. Nothing wrong, but nothing special – The page achieves its purpose; however, it  does not merit a High quality rating, nor is there anything to indicate that a Low quality rating is appropriate.
  2. Mixed with strong High quality rating characteristics – The page or website has strong High quality rating characteristics, but also has mild Low quality characteristics. The strong High quality aspects make i t difficult to rate the page Low.

9.0 Page Quality Rating Tasks

Page quality rating tasks might seem difficult at first but with this guideline, in place, we recommend that you read it a couple of times and refer to sections so that you can develop a total understanding of what is required from a search quality evaluator. But do understand that these guidelines do not cover each aspect of every page out there, you’re required to develop your acumen and you should rate a page high or low quality depending on your understanding as well as the requirements of your locale.

9.1 Instructions for Rating Page Quality Tasks

Page quality tasks are usually broken down into four major steps. But these are not limiting and can be reduced to increase depending on the requirement of the task, regional requirements, and specific business requirements.

  1. Some initial questions about the task landing page.
  2. A “PQ grid”  to record your observations about PQ characteristics of the landing page.
  3. The Overall PQ rating slider which records your Overall PQ rating.
  4. A comment box to explain your rating.

9.2 Reputation and E-A-T (expertise, authoritativeness and trustworthiness):

Website or the Creators of the Main Content?

You must evaluate the expertise, authoritativeness, and trustworthiness of a website as well as the credibility and repute of the creators of its MC(Main Content). This is especially important when different pages on the website have different creators.

10.0 Page Quality Criteria for Specific Types of Pages

10.1 Ratings for Encyclopedia Pages

We may not always know the author of the specific encyclopedia article, and therefore must rely on website reputation research to determine the E-A-T of the article. High and Highest quality ratings should only be used for encyclopedias with very good reputations for accuracy and expertise, where the article itself is well-researched with appropriate references cited.

10.2 Ratings for Pages with Error Messages or No MC

Some pages are temporarily broken pages on otherwise functioning websites, while some pages have an explicit error (or custom 404) message. In some cases, pages are missing MC as well. Please think about whether the page offers help for users—did the webmaster spend time, effort, and care on the page?

10.3 Ratings for Forums and Q&A pages

For this category keep the following points in mind :

  1. The Main Content on forum and Q&A pages includes both the question as well as the answers/responses and resulting discussions.
  2. Rate forum and Q&A pages from the point of view of a user who visits the page, rather than a participant involved in the discussion.

11.0 Page Quality Rating FAQs

 Page Quality Rating FAQs

Part 2: Understanding Mobile User Needs

12.0 Understanding Mobile Users, Mobile Queries, and Mobile Results

People rely on smartphones to do a lot for themselves nowadays. But at the same time we face some challenges that we don’t face while using a desktop computer or laptop:

  1. Entering data may be cumbersome: typing is difficult on mobile smartphones, and when users speak to their phones instead of typing, voice recognition may not always be accurate.
  2. Small screen sizes make it difficult to use some phone  features, apps, and webpages.
  3. Some webpages are difficult to use on a mobile phone. Website navigation can be difficult as menus and navigation links may be small. Webpages may require left-to-right scrolling to read text. Images may not fit on the screen. In addition, many mobile devices cannot access webpages with Flash or other similar features.
  4. Internet connectivity can be slow and inconsistent for mobile users going in and out of networks. App opening, recognition of voice commands, and webpage load times can be very slow on a mobile phone.

12.1 Important Rating Definitions and Ideas

  1. Query: This refers to the word(s) and/or number(s) that a user types or speaks into a mobile phone.
  2. User: The user is the person trying to find information or accomplish a task by typing or speaking into a mobile phone with a small screen (i.e., size of a smartphone, not a tablet).
  3. User Intent: When a user types or speaks a query, he or she is trying to accomplish something. We refer  to this goal as the user intent.
  4. Locale: All queries have a locale, which is the language and location f or the task. Locales are represented by a t wo-letter country code.
  5. User Location: This tells us where the user is located and should be inferred from the map provided.
  6. Search Engine Results Page (SERP): The page a search engine shows after a user enters a query in the search box.
  7. Result: We will use the word result to refer to the result block and the landing page.
  8. Device Actions: Mobile phones and other devices can respond to voice commands to perform many actions, such as setting an alarm or opening an app.

12.2 Understanding the Query

Understanding the query is the first step in evaluating the task. Remember, a query is what a user types or speaks into a mobile phone.

12.3 Locale and User Location

All queries have a task language and task location (referred to in rating tasks as the “Locale”). The locale is important for understanding the query and user intent. Users in different locations may have different expectations for the same query.

12.4 Queries with an Explicit Location

Sometimes users tell search engines exactly what kinds of results they are looking for by adding the desired location in the query, regardless of their user location. We’ll call this location inside the query the “explicit location.” The explicit location makes queries much easier to understand and interpret.

12.5 Queries with Multiple Meanings

Many queries have more than one meaning. For example, the query [ apple] might refer to the computer brand or the fruit.

We will call these possible meanings query interpretations.

  1. Dominant Interpretation: The dominant interpretation of a query is what most users mean when they type the query. Not all queries have a dominant interpretation. The dominant interpretation should be clear to you, especially after doing a little web research.
  2. Common Interpretation: A common interpretation of a query is what many or some users mean when they type a query. A query can have multiple common interpretations.
  3. Minor Interpretations: Sometimes you will find fewer common interpretations. These are interpretations that few users have in mind. We will call these minor interpretations .

12.6 Query Meanings Can Change Over Time

Remember to think about the query and its current meaning as you are rating. We will assume users are looking for current information about a topic, the most recent product model, the most recent occurrence of a recurring event, etc. unless otherwise specified by the query.

12.7 Understanding User Intent

It can be helpful to think of queries as having one or more of the following intents.

  1. Know query, some of which are  Know Simple queries
  2. Do query, some of which are Device Action queries
  3. Website query when the user is looking for a specific website or webpage
  4. Visit-in-person query, some of which are looking f or a specific business or organization, some of which are looking f or a category of businesses

12.8 Understanding Result Block

There are different types of Result Block for every query. To interpret them individually based on requirements.

 12.9 Rating on Your Phone Issues

We understand that raters using different phones, operating systems, and browsers may have different experiences. In general, do what you would do naturally, and rate based on your experience.

Part 3: Needs Met Rating Guideline

13.0 Rating Using the Needs Met Scale

Rating Meter:   

needs-met-scale
  1. FullyM: Fully meets almost all users’ requirements exclusively.
  2. HM:       Highly meets most users’ requirements.
  3. MM:      Moderately meets means it’s either helpful for many or very helpful for some.
  4. SM:        Slightly meets means many or most users would like to see additional results.
  5. FailsM: Fails to meet any user requirements.

Rating may be assigned in between any of the two labels as well.

13.1 Rating Result Blocks: Block Content and Landing Pages

Each result needs to be rated for Needs Met Rating. The part that you rate would depend upon both the query and result block.

Special Content Result Block: This should play the largest role in your rating. Some SCRBs may have links to landing pages.

If most users would not click then the rating should be based on block content alone.

If users may click on it then the helpfulness of landing pages is also considered in rating.

Web Search Result Block: Needs a click and landing pages should be evaluated.

Device Action Result Block: Rated on the helpfulness of the action.

For example, What does Love mean

SCRB:

scrb-example

Rating based on the content inside the block as there are no obvious landing pages.

Web Search Result Block:              

example-of-search-block

Rating based on the content of the landing page as the user would need to click on it.

Some more examples where rating should be based on the content inside the block are as under.

one-more-example-of-landing-page

Note: This section applies to Needs Met ratings. For SCRBs that have landing pages you may be asked to provide page quality ratings and for that refer to section 14.0.

13.2 Fully Meets (FullyM)

It’s a special rating category used when

  1. The query and user need should be completely unambiguous.
  2. Mobile users should be satisfied and require minimum effort for the query’s answer.
  3. All or almost all users would be completely satisfied.

FullyM should be reserved for complete and perfect responses. Some scenarios where FullyM is appropriate are.

  1. When users find specific webpage they were looking for.
  2. When result block completes a device action required by the user.
  3. When user finds a specific fact that they were looking for. Accuracy of the fact should be checked before rating FullyM.

It may apply in other situations as well. Go for a lower rating when in doubt.

Note: If the result block is almost FullyM but may need additional info HM rating is more appropriate.

Sometimes the specificity of the query may also need to be considered.

13.2.1 Examples of Fully Meets (FullyM) Result Block

first-example-of-fully-meets-results-block
second-example-of-fully-meet-results
third-example-of-fullyM
fourth-example-of-fully-meets-blocks
fifth-example-of-fullym

13.2.2 Examples of Queries that Cannot Have Fully Meets Results

queries-that-cannot-have-fully-meets

13.3 Highly Meets

It meets the requirements of most users and results are a good fit for the query. Information pages like encyclopedias must be highly credible to fall in this category. Scientific information pages must conform to the consensus.

A single query may have many HM results. It is especially important for queries that have multiple on-topic results.

13.3.1 Examples of Highly Meets (HM) Result Blocks

first-example-of-highly-meet-results
second-example-of-highly-meet-results
example-of-highly-meets
third-example-of-highly-meet-query
fourth-example-of-highly-meets

13.4 Moderately Meets (MM)

These are results that are satisfying for many users or highly satisfying for some users. Results should still fit the query but they have fewer qualities than HM.

These results are not inaccurate and are generally average to good.

13.4.1 Examples of Moderately Meets (MM) Results Block

moderate-meet-results-example
moderate-meet-example

13.5 Slightly Meets (SM)

Results that are helpful for a few users. It may have some minor inaccuracies or be outdated. Results may also be too specific or too broad for a higher rating.

Note: Mobiles have higher costs per click and the clicks depend upon titles. Exaggerated or, is leading titles may also be SM or below.

13.5.1 Examples of Slightly Meets (SM) Result Blocks

first-example-of-sm-blocks
part-of-example-of-slightly-meet
second-example-of-slightly-meets
highly-meet-results-query
third-example-of-highly-meets

13.6 Fails to Meet (FailsM)

first-example-of-fails-meet
second-example-of-fails-to-meet
part-of-example-offails-to-meet
third-exampleof-fails-to-meet
third-example-part-of-fails-to-meet-query

These results are helpful to either very few or no users at all. Results may be unrelated to the query or incorrect. It fails to meet user requirements or may be completely outdated information.

Results should never be offensive or unpleasant.

  1. Harmful to Self or Other Individuals (Section 7.1)
  2. Harmful to Specified Groups (Section 7.2)
  3. Harmful Misleading Information (Section 7.3)
  4. Untrustworthy (Section 7.4)
  5. Spammy (Section 7.5)
  6. Porn (Section 15.1)

13.6.1 Examples of Fails to Meet (FailsM) Result Blocks

14.0 The Relationship between Page Quality and Needs Met

Needs Met Rating depends upon both query and results

needs-met-rating-scale-for-both-search-and-query

Page Quality rating does not depend on the query

scale-for-page-quality

Page quality reading only needs to be given when there is a slider and only depends upon the quality of the content.

Some Guidelines are as under

  1. Useless results should be rated FailsM even if page quality is good.
  2. Lower content quality should mean lower Needs Met Rating as well.
  3. HM rating should be given to helpful, high quality pages. Medium quality pages may be rated HM if other qualities are there.
  4. Low page quality does not merit an HM rating.
  5. Low quality on-topic pages should be SM. If the content is completely gibberish it should be FailsM. The only exception is clear website intent in the query where search result should be rated FullyM regardless.
  6. If the page serves no purpose it should always be rated Lowest page quality.
example-of-fails-m-page
example-of-quality-pages
example-of-high-quality-pages

15.0 Rating Porn, Foreign language, and Did Not Load Results

Assignment of flags does not depend on the query. Screenshot of flags

screen-shot-of-flag-section

Example

assignment-of-flags-section

15.1 Porn Flag

Porn needs to be flagged as porn even if the query is specifically asking for porn.

15.2 Needs Met Rating For Porn Results

15.2.1 Needs Met Rating for Clear Non-Porn Intent Queries

FailsM will be assigned if a query is non-porn related.

For example

this-is-an-example-of-non-porn-intent

15.2.2 Needs Met Rating for Possible Porn Intent Queries

Some queries may have both porn and non-porn intended meanings. They will be rated as though the query is non-porn-related.

Example:

example-of-possible-porn-intent

15.2.3 Needs Met Rating for Clear Porn Intent Queries

Queries that have clear porn intents should be rated as per relevance to the query. But the page is still to be flagged as porn

this-snap-shot-is-an-example-of-clear-porn-intent-query

15.3 Reporting Illegal Images

Child Pornography

Note: Federal laws of task location need to be followed.

Definition of Child Pornography

If an image depicts someone minor or appears to be a minor in sexually explicit conduct it is called child pornography. It may contain actual minor or computer-generated imagery. Even drawings, cartoons, anime, or sculptures depicting sexually explicit conduct for minors are child pornography.

Even if the pornographic images depict children in a literary, political, artistic, or scientific context, please forward them to your employer. URLs are required to be forwarded as instructed.

Depiction of minors’ genitals does not require them to be uncovered. They can be considered Child pornography even if covered. A person pretending to be a minor in sexually explicit content is not child Pornography but if you cannot tell whether the person being shown in such a scene is over the legal age then it is considered Child Pornography.

15.4 Foreign Language Flag

15.4.1 Using the Foreign Language Flag

If the language of the result is not the task language or the language used by the majority in the task location it is to be flagged as Foreign.

this-snap-shot-showing-foreign-language-flag

Important

  1. Assign the flag even if you understand the foreign language.
  2. Flag should be assigned based on the content of the landing page.
  3. Flag even if a foreign language page is targeted by the query.
  4. When In doubt don’t use the foreign language flag.

15.4.2 Needs Met Rating for Foreign Language Results

Most Foreign Language pages would be rated FailsM as people in the task location wouldn’t understand it, unless a query specifically asks for a foreign page, then assign FullyM.

Note: If you can’t read the foreign language content page you don’t have to assign a page quality rating.

foreign-language-results

15.5 Did Not Load Flag

15.5.1 Using the Did Not Load Flag

The flag is used when an error message load or the landing page is completely blank. The flag is assigned on the loading page, not the result block.

For Example

load-Flag

A flag is not assigned for:

  1. Malware warnings.
  2. Pages that were removed or expired.
  3. Inaccessible due to required subscription.

15.2.2 Needs Met Rating and the Did Not Load Flag

Did not Load pages are also assigned FailsM as they are useless. If the page partially loads, the Needs Met Rating is assigned based on the available content. Page Quality Rating is not assigned if a page cannot be evaluated.

15.6 Additional Flags in Some Rating Tasks

 Some rating tasks may ask you to rate something that might be upsetting or offensive. Such rating is assigned based on the typical user of your locale.

16.0 Rating Queries with Multiple Interpretations and Intents

Some queries may have multiple interpretations. Conform to the following guidelines while dealing with them:

  1. Very helpful result for dominant interpretation is HM some might even be FullyM.
  2. Very helpful result for a common interpretation is HM or MM.
  3. Very helpful result for a Minor interpretation is SM or lower.
  4. No chance interpretations are FailsM.

16.1 Rating Queries with Both Website and Visit-in-Person Intent

Users may have dual intent of either visiting a website or visiting some store in-person. If the intent is not clear then results satisfying only one of the intents is never rated Fully Meets.

example-of-rating-queries
example-of-rating-query

17.0 Specificity of Queries and Landing Pages

example-of-query-and-landing-page
second-example-of-query-and-landing-page
third-example-query-landing-page
fourth-example-of-query-and-landing-page

Queries may be specific or general as shown in the table, but the rating should always depend upon how helpful the result is. If the query is a broad category then the most popular results according to locale should be very helpful.

18.0 Needs Met Rating and Freshness

Some queries require very recent information.

first-example-of-need-met-rating

Some queries demand breaking news. This is when old pages will be rated FailsM. Some queries may be looking for recent or timeless information. Page Quality rating is generally not based on freshness unless the page has been abandoned.

Note: Websites show different dates depending on their settings, if you want to know the date of the content, try the “Wayback Machine” on the Internet Archive.

second-example-of-need-met-query
third-example-of-needs-met-rating
fourth-example-of-needs-met-rating

19.0 Misspelled and Mistyped Queries and Results

19.1 Misspelled and Mistyped Queries

example-of-misspelled-and-mistyped-queries

If queries are obviously misspelled, rate the results on the user intent. For queries that are not very obvious in misspelling, rate the results as per the user spellings.

19.2 Name Queries

Follow the given examples

example-of-name-queries

20.0 Non-Fully Meets Results for URL Queries

URL queries may be issued by some users to find info about the website so the rating for any result that is not the website itself may not be FailsM. Usage statistics, however, are not what most users are looking for.

21.0 Product Queries: Importance of Browsing and Researching

User queries may have the intent to buy or seek information and they may be unspecified as well. Users also enjoy online browsing and need authoritative information on the products while scouting them online. You have to keep in mind all these factors while assigning Needs Met Rating to results. Results for product queries may be important for both your money and your life (YMYL.)

example-for-product-query
second-example-product-query

22.0 Rating Visit-in-Person Intent Queries

Users can travel different range of distances while looking for different services. For example, they may be willing to travel farther for a clinic than for a famous coffee shop. So you have to use your judgment on the meaning of “nearby”.

22.1 Examples Where User Location Does (and Does Not) Matter

first-example-of-user-location
second-example-of-user-location
user-location-example

23.0 Rating English Language Results in Non-English Locales

Even if you can understand English, the Needs Met Rating is supposed to be based on the users in your Locale. Unless a query specifically asks for results in English, for a non-English locale, results in English should be considered useless.

Important: There may be many languages to consider including task language, official language, local dialects, etc. When you face doubt, prefer the results in task language unless the query specifically indicates otherwise.

23.1 Examples of English (and Non-English) Results in Non-English Locales

results-in-non-english

Here are some examples that include proper nouns in non-English queries.

example-of-non-english-queries

Here are some examples where users would still prefer results in the local language.

example-of-local-language-results

There may be queries where English language results would satisfy the user more even if the query is in task language. You will have to use your judgment for assigning ratings.

task-language

In locales where English is the official language, results in English would be preferred.

query-results-in-english

24.0 Rating Dictionary and Encyclopedia Results for Different Queries

You have to judge whether a user is looking for the meaning of a word or the word is commonly known and intent may be different from Dictionary results. You should mostly reserve Dictionary results for queries that include “What is” or “What is the meaning of.”

Important: If a commonly known word is used in the query, an SM rating may be appropriate for a dictionary result.

example-of-rating-dictionary
rating-dictionary-example
author

Farasat Khan

Khan is an SEO consultant with 8 years of experience. His expertise has helped hundreds of brands generate additional revenue through search traffic. He is an SEO cum Digital Marketing specialist specializing in content strategy, search intent, and the sales funnel.

Recent Blog
January 23 18 minutes READ

Content marketing allows SaaS businesses to build trust and authority. By creating valuable and consistent content, you can attract and engage potential customers at every stage of the buying journey. Follow our guide to build an effective content marketing strategy for your software.

January 03 9 minutes READ

As of Dec 2022, the total market size of SaaS businesses is estimated to be around $272.5B. Many entrepreneurs and marketers are looking for ways to stay up-to-date with the latest trends in SaaS technology, strategies, and marketing. One great way to do that is to follow some of the best SaaS blogs out there. ... Search Quality Evaluators Guidelines – Brief

January 01 19 minutes READ

SEO is crucial for software companies; 80% of the decision-makers chose to educate themselves from informational articles rather than targeted ads. SEO can help SaaS companies reach their target audiences more quickly and effectively than ever before, helping them grow their user base rapidly.