The questions are the product

How a question set gets built

A question set is not a translation exercise. The same commercial need is expressed differently in Riyadh and in Cairo, differently by a procurement officer and by a technical lead, and differently again depending on whether the buyer already knows which companies exist. Getting this wrong produces a clean report that measures nothing.

1. i.

   Understand how the sector actually buys

   Who initiates, who evaluates and who signs differ by industry. A distributor is chosen by a reseller's commercial team on credit terms and vendor coverage. A private label manufacturer is chosen by a foreign sourcing manager on certifications and minimum order quantities. A listed contractor is researched by analysts who care about backlog and governance. Each of those buyers types a different question, so each gets a different set.

2. ii.

   Separate discovery from verification

   A buyer who does not yet know which companies exist asks the assistant to produce candidates. A buyer who already has your name asks whether you are any good. Those two produce completely different results, and blending them into one average hides the finding. Most companies score well on the second and badly on the first, which is exactly the problem worth paying to fix.

3. iii.

   Write each question in four registers

   Modern Standard Arabic for formal and Gulf corporate contexts, Gulf spoken phrasing, Egyptian spoken phrasing, and English. A procurement officer in Saudi Arabia does not type the same words as a factory manager in the Delta. Where a sector uses English technical vocabulary inside an Arabic sentence, which is common in technology, the question is written the way it is really typed rather than the way a purist would write it.

4. iv.

   Test and discard

   Draft questions get trial runs first. Any question that returns a definition rather than a list of companies is cut, because it measures nothing commercial. Any question that returns the same three companies every time regardless of phrasing is kept, because it is the one that decides deals.

5. v.

   Fix the wording and never change it again

   Once a set is agreed the wording is frozen, because a re-test six weeks later is only meaningful if the question is identical. Changing a single word between the baseline and the re-test invalidates the comparison, which is a mistake that is easy to make and impossible to detect afterwards.

What a finished measurement looks like

The sample below is from an audit of a Gulf technology distributor. Six discovery questions were used, meaning questions from a buyer who does not yet know which companies exist. Each question was asked twenty times in Arabic and twenty times in English, spread across ChatGPT, Gemini and Perplexity, every run from a fresh session with no account signed in. The figures are illustrative.

Three of the six questions, in full

All six questions together

Across all six discovery questions, 120 runs in each language. Shown as the share of runs that named each company, so the two languages can be compared directly.

Company	Named, English	Named, Arabic	Change
Distributor A	80%	73%	▼7 points
Distributor B	68%	65%	▼3 points
Distributor C	60%	52%	▼8 points
The client	58%	17%	▼41 points

Every competitor in the test loses a few points between languages, which is what the thinness of Arabic material on the web would predict. The client loses forty one. That difference is the finding, and it matters because a problem affecting the whole category cannot be fixed by one company while a problem specific to one company can.

A shortlist holds four names. In English the client is fourth. In Arabic it is sixth.

The same audit, asked who runs the company

One further question was added outside the six, asked twenty times in each language.

Answer returned	Runs	Share
Named the previous chief executive	23 of 40	57%
Named the current chief executive	11 of 40	28%
Declined or hedged between the two	6 of 40	15%

The cause was a single page. The company's own About page still named the outgoing chief executive while its investor announcement named the incoming one, and a company's own domain carries heavy weight when an assistant is asked to state a fact about that company. Two hours of work moved this to above eighty five percent within a month.