#Search by image

Identify products in a picture and find matching or similar products in the catalog.

##What this endpoint does

Endpoint

```
POST https://mcp.gaisho.co/v1/tools/search_products_by_image
```

Find catalog products from a photo, screenshot or product image. A picture can contain several objects; the response identifies them and searches for the selected one. Matching tiers distinguish the same item from other sizes, variants and substitutes.

• -Find a product from its packaging or a shopper’s photo.
• -Research an item shown in a shopping screenshot.
• -Search a second object in an already-submitted picture without uploading it again.

##Before you call

• -Supply exactly one source: a public image_url, a base64 image_data_url, or a previously returned image_id. A private browser or file-system URL is not a downloadable image URL.
• -For image_data_url, use JPEG, PNG, GIF or WebP and a decoded image of at most 5 MB. Use the data:image/...;base64,... prefix.
• -object_index is zero-based; 0 is the most prominent object. Inspect objects before selecting another index. An index beyond the objects read returns INVALID_ARGUMENT naming the valid range.
• -verify defaults to true. With false, candidate verification is skipped; a listing whose own image is the picture (same_image) can still have tier same, while other unchecked candidates remain unjudged. Do not describe verification as completed. Submitted images and results are retained for 30 days; image_id can be reused during that period.

Send a JSON object as the request body and authenticate with Authorization: Bearer <API key>. Amounts are in USD unless the result explicitly says otherwise.

##Request body

##Example request

Set GAISHO_API_KEY in your server or local environment. These examples use a JSON POST request; Python uses the standard library and JavaScript uses Node.js fetch. The 60-second client timeout in Python and JavaScript is configurable, not a service latency guarantee.

cURL

```
curl --fail-with-body 'https://mcp.gaisho.co/v1/tools/search_products_by_image' \
  -H "Authorization: Bearer $GAISHO_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "image_url": "https://example.com/product-photo.jpg"
}'
```

Python

```
import json
import os
import urllib.error
import urllib.request

arguments = json.loads(r'''{
  "image_url": "https://example.com/product-photo.jpg"
}''')
request = urllib.request.Request(
    "https://mcp.gaisho.co/v1/tools/search_products_by_image",
    data=json.dumps(arguments).encode("utf-8"),
    headers={
        "Authorization": "Bearer " + os.environ["GAISHO_API_KEY"],
        "Content-Type": "application/json",
    },
    method="POST",
)
try:
    with urllib.request.urlopen(request, timeout=60) as response:
        payload = json.load(response)
    print(json.dumps(payload, indent=2))
except urllib.error.HTTPError as error:
    print(error.read().decode("utf-8"))
    raise
```

JavaScript

```
// Node.js 18+; keep this code and the key on your server.
const response = await fetch("https://mcp.gaisho.co/v1/tools/search_products_by_image", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.GAISHO_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
  "image_url": "https://example.com/product-photo.jpg"
}),
  signal: AbortSignal.timeout(60_000),
});
const body = await response.text();
if (!response.ok) {
  throw new Error(`HTTP ${response.status}: ${body}`);
}
const payload = JSON.parse(body);
console.log(payload);
```

##Common request patterns

###Search another object in the same picture

Use a valid returned image ID and an index present in objects.

JSON request body

```
{
  "image_id": "IMAGE_ID_FROM_PREVIOUS_CALL",
  "object_index": 1,
  "verify": true
}
```

###Retrieve unverified candidates

Suitable for an exploratory shortlist; inspect or verify candidates before calling them exact matches.

JSON request body

```
{
  "image_url": "https://example.com/product-photo.jpg",
  "verify": false
}
```

##Response fields

The HTTP response contains tool and result. All fields below the envelope, including any assessment summary, are inside result.

##Example response

Examples are illustrative and may omit optional detail fields. Prices are not live quotes; identifiers and example.com URLs are placeholders to replace, not usable product records.

HTTP 200 · illustrative, shortened

```
{
  "tool": "search_products_by_image",
  "result": {
    "id": "IMAGE_ID_FROM_PREVIOUS_CALL",
    "objects": [
      {
        "label": "cordless vacuum",
        "brand": "Example",
        "product": "cordless stick vacuum",
        "model": "V1",
        "variant": "black",
        "visible_text": "EXAMPLE V1",
        "gtin": null,
        "queries": [
          "Example V1 cordless vacuum"
        ],
        "similar": "black cordless stick vacuum",
        "box": [
          120,
          40,
          880,
          980
        ]
      }
    ],
    "object": 0,
    "items": [
      {
        "product_id": "PRODUCT_ID_FROM_SEARCH",
        "title": "Example cordless vacuum",
        "brand": "Example",
        "image": "https://example.com/product-photo.jpg",
        "price": 179.99,
        "retailer": "example.com",
        "offer_id": "OFFER_ID_FROM_SEARCH",
        "url": "https://example.com/vacuum",
        "tier": "maybe",
        "p_same": 0.65,
        "same_image": false,
        "gtin_match": false
      }
    ],
    "counts": {
      "maybe": 1,
      "different": 6
    },
    "verified": true,
    "message": null,
    "errors": null
  }
}
```

##How to read the result

• -REST puts the image payload in result. id identifies the stored image; object is the selected object’s integer index, not the object details themselves.
• -items contains candidates. same identifies a same-item match, including a listing whose own image is the picture (same_image is true); maybe is a possible match that still needs checking. Neither guarantees details that are not visible in the image. Read product and listing details before buying.
• -size differs only in size or quantity; compare unit prices and the requested quantity. substitute is an alternative for the same purpose, not the same item. variant is another version in the same product line, such as a different color or bundle. unjudged has not been assessed and should not be presented as a confirmed match.
• -verified reports whether the requested verification completed; it does not mean every returned candidate is identical. Check the candidate’s own tier.
• -errors can accompany usable partial results. counts includes classified candidates before the display limit and can include different (unrelated products), which is excluded from items. Counts therefore need not add up to the displayed list length. Pass an item’s product_id to get_product for specifications and offers.

##Empty results and errors

A successful request can return no candidates, missing information or an unavailable assessment. Check the result before recommending a product. For request failures, use the error code and message; retrying the same invalid input will not fix it.

• -[Error handling and retries](https://gaisho.co/docs/api#errors): HTTP statuses, structured errors, rate limits and quota responses.

##What to call next

• -[Get product details](https://gaisho.co/docs/api/get_product): Read product specifications, ratings, reviews and offers, for one product or up to eight at once.
• -[Get offer details](https://gaisho.co/docs/api/get_offer): Read a listing’s purchase URL, seller, price, coupon terms and stated product information.

• -[Use this tool through MCP](https://gaisho.co/docs/mcp/search_products_by_image): The same capability through the other connection.
• -[REST API conventions](https://gaisho.co/docs/api): Authentication, response envelopes and request handling.