LVISnet wholesale price import#

LVISnet is the joint product and price catalogue of Finnish HVAC and electrical wholesalers. ELT uses it for real-time wholesale prices in the installer’s PWA product search: when an installer scans a barcode on site, they see the right wholesale price and availability.

We support these wholesalers: Onninen, Dahl, Ahlsell, Sonepar.

How the import works#

Field Service → LVISnet → Fetch Sources lists the wholesalers. Each row has:

  • Active flag: whether LVISnet should be fetched automatically?
  • Last run state and timestamp
  • Pricelist prices are stored to
  • Wholesaler link (the wholesaler on the Odoo side)

Automatic fetch is the 1st of the month at 21:25 (_cron_fetch_all). Fetch manually if LVISnet announces an out-of-band price update:

  1. Field Service → LVISnet → Fetch Sources.
  2. Click the Onninen / Dahl / Ahlsell / Sonepar row.
  3. Fetch now button.
  4. The wholesaler is marked “fetch_requested”; the dispatcher cron (every minute) picks it up and starts.

What one fetch does#

  1. Lists the OpusCapita catalogue (REST API) → list of zip files.
  2. Downloads the zip (e.g. OLTUOTE.zip — Onninen Product file).
  3. Opens and finds the inner TXT file in OVT format.
  4. Detects the type:
    • HINTA.txt → prices → goes into the pricelist’s pricelist_item
    • TUOTE.txt → product info → goes into product_template
  5. Imports row by row, commits every 1000 rows (since 200.14).

The import is a big operation: a single wholesaler’s TUOTE file can be 170,000+ rows, takes ~2 hours the first time. Once most products are already in the database, daily runs are faster (~20–30 min) because many rows are UPDATEs not CREATEs.

Progress monitoring#

Field Service → LVISnet → Fetch Runs:

ColumnWhat
SourceWhich wholesaler
Staterunning / ok / error / partial
Started / FinishedTime range
Files importedHow many files succeeded
SummaryReal-time progress, e.g. “Importing OLTUOTE.txt: 87,000 / 171,540 rows” — updates every ~40 s

Summary is new (200.15) — earlier the state just said “running” for an hour or two, with no signal whether things were progressing or stuck.

Watchdog#

If an import hangs for over 4 hours without progress (typically: worker process killed via SIGKILL, container restart), the _unstick_stale_runs watchdog automatically marks it as error and reopens other wholesalers for re-run.

Previously (60 min watchdog) this killed healthy long imports — now 4 h gives sufficient headroom (200.15).

Daily check#

In the morning:

  1. Open Fetch Runs, filter to “Today”.
  2. Verify all 4 wholesalers are in ok state.
  3. If something is in error, click → Summary tells you why:
    • Stuck for >240min → investigate Odoo locking
    • XML parse error → reported via support
    • Connection refused → LVISnet is down, retry later
  4. If partial, some file succeeded but another didn’t. Try re-running with Fetch now.

Product and price visibility#

After an import:

  • Products appear in Inventory → Products
  • Wholesale prices appear in each product’s Pricelists tab
  • Availability is in the lvisnet.product.wholesaler.stock rows (per product × wholesaler)
  • PWA product search syncs automatically: the installer sees the correct wholesale price when scanning a barcode

Troubleshooting#

Import gave ’lock timeout'
Most common cause: another deploy / cron is using the same `product_template` table at the same time. **200.14's per-batch commit** fixed the worst situations, but if the issue recurs: - **Are other elt_fsm deploys running right now?** - **Is another LVISnet fetch running?** (latest improvement: cron prevents concurrents)
Same EAN code on two products
Earlier this killed the entire import. **200.13's fix**: try/except that leaves the barcode field off the row that hits the unique constraint. Result: the product is created without barcode, the other product keeps the EAN. The data still has old duplicates from earlier runs. Cleanup is planned — ask the developer to schedule.
A product is missing from the PWA’s search
- **PWA cache offline?** Installer forces: Settings → "Sync inventory now". - **Product new in LVISnet?** Check the import ran after the addition. - **Product archived (`active=False`)?** Search Products list with "Archived" filter.