A major architectural rewrite: the metal/chalcogen space now spans 3d–5d transition metals and O/S/Se/Te chalcogens, automated PDF table mining feeds the training set directly from the literature, and every prediction now ships with a 90% bootstrap confidence interval.
3d, 4d, 5d metals × O, S, Se, Te
27 physics descriptors (v4.0 had ~12)
Auto-extracts compound data from papers
90% bootstrap confidence intervals
Version 6.0 of the MₓCᵧ pipeline is less an update than a rewrite. Where v4.0 worked with a 12-feature set restricted to six 3d transition metals and three chalcogens, v6.0 extends the periodic table coverage to 3d, 4d, and 5d metals against O, S, Se, and Te — 23 metals × 4 chalcogens — and nearly doubles the feature set to 27 physics-informed descriptors. The two headline additions are an automated PDF table-mining stage that feeds literature data directly into the training set, and a bootstrap-calibrated 90% confidence interval attached to every band gap and magnetisation prediction.
v4.0's stacking ensemble treated all compounds uniformly and gave a single point estimate per property. In practice, oxide insulators (charge-transfer, large U) behave very differently from sulfide/selenide/telluride insulators (smaller U, more covalent), and a single-point gap prediction without an error bar is not publication-ready. v6.0 directly addresses both issues.
1. Element Space — From 6 Metals to 23
v4.0's ATOMIC_DATA table covered only Ti, V, Cr, Mn, Fe, Co, Ni, Cu —
the 3d block. v6.0 adds the full 4d row (Zr, Nb, Mo, Ru, Rh, Pd, Ag) and 5d row
(Hf, Ta, W, Re, Os, Ir, Pt), plus oxygen as a fourth chalcogen alongside S, Se, Te.
Each row now carries six physical constants instead of six, with the addition of
reliable ionic radii from Shannon (1976).
| Block | Elements | d-electron range | Why it matters |
|---|---|---|---|
| 3d | Ti–Zn | n_d = 2–10 | Strong correlation (large U/W) — Mott physics dominant |
| 4d | Zr–Ag | n_d = 2–9 | Larger orbital extent → smaller U, more itinerant |
| 5d | Hf–Pt | n_d = 2–8 | Strong spin-orbit coupling (λ up to 1800 meV) — relativistic effects matter |
ATOMIC_DATA = {
# 3d: [n_d, r_cov, chi, IE, U_Hubbard, r_ionic]
"Fe": [6, 1.26, 1.83, 7.90, 4.5, 0.92],
# 4d
"Ru": [6, 1.46, 2.20, 7.36, 3.0, 0.82], # U is smaller than Fe
# 5d
"Os": [6, 1.44, 2.20, 8.44, 3.0, 0.77], # same n_d as Fe, very different physics
}
# New: spin-orbit coupling constant λ (meV), grows steeply down a column
SOC_LAMBDA = {"Fe": 50, "Ru": 300, "Os": 1200} # Fe→Ru→Os: 24× increase
2. The 27-Feature Set — What's New vs v4.0/v5.1
The previous version used roughly 12 features built around crystal field splitting, the Mott U/W ratio, and electronegativity difference. v6.0 keeps all of those and adds five new physics-derived descriptors that specifically target the 4d/5d expansion and known failure modes of the v4.0 model.
| New feature (v6.0) | Formula | Physical role |
|---|---|---|
| zsa_delta | Δ = ε_d − ε_p − U/2 | Zaanen–Sawatzky–Allen charge-transfer energy. Δ<0 → charge-transfer insulator; Δ>0 → Mott-Hubbard insulator. Distinguishes the two insulating mechanisms the v4.0 model conflated. |
| soc_sq | λ² / 10⁶ | Second-order spin-orbit coupling — becomes significant for 5d metals (Os, Ir, Pt) where λ > 1 eV, negligible for 3d. |
| stoner_split | I × W | Stoner exchange splitting energy — predicts itinerant ferromagnetism when exceeding the Stoner criterion (I·N(E_F) > 1). |
| d_fill_ratio | n_d / 10 | Normalised d-band filling — lets the model learn smooth trends across the whole 3d–5d series rather than treating n_d as a categorical variable. |
| goldschmidt_t | (r_M+r_C) / [√2(r_M+r_O)] | Ionic-radius tolerance factor adapted from perovskite stability theory — flags compounds likely to adopt distorted, lower-symmetry structures. |
v5.1 included a "gap_log_proxy" feature that was found to be circular — it leaked information derived from the target itself during testing. It was deliberately removed in v6.0; CV scores dropped slightly (honestly) but test-set generalisation improved.
3. New: Automated PDF Table Mining (Cell 6b)
This is the most significant workflow change. Previously, adding literature data
meant manually typing rows into PAPER_DB_MANUAL. Cell 6b now scans
an entire directory of PDFs, extracts text with a poppler/pypdf fallback chain,
and uses a regex-based formula-and-numbers scanner to propose
(compound, band_gap, magnetization) candidate rows automatically.
How the formula scanner works
_FORMULA_PAT = re.compile(r'\b([A-Z][a-z]?)(\d{0,2})([A-Z][a-z]?)(\d{0,2})\b')
_NUM_PAT = re.compile(r'-?\d+\.\d+|-?\d+')
_KNOWN_METALS = {"Ti","V","Cr","Mn","Fe","Co","Ni","Cu","Zn",
"Zr","Nb","Mo","Ru","Rh","Pd","Ag",
"Hf","Ta","W","Re","Os","Ir","Pt"}
_KNOWN_CHALCOGENS = {"O","S","Se","Te"}
def extract_tables_from_pdf(pdf_path):
"""Scan every line for formula + ≥2 trailing numbers in physical range."""
text = _read_pdf_text(pdf_path)
records = []
for line in text.splitlines():
fm = _FORMULA_PAT.search(line)
if not fm: continue
M_cand, C_cand = fm.group(1), fm.group(3)
if M_cand not in _KNOWN_METALS or C_cand not in _KNOWN_CHALCOGENS:
continue
nums = [float(n) for n in _NUM_PAT.findall(line[fm.end():])]
if len(nums) < 2: continue
gap, mag = nums[0], nums[1]
# Physical plausibility gate — rejects page numbers, years, etc.
if not (0.0 <= gap <= 12.0) or not (0.0 <= mag <= 20.0):
continue
records.append({"compound":..., "band_gap_eV":gap, "magnetization_muB":mag, ...})
return records
Cell 6b deliberately produces candidate rows for visual review,
printing the matched compound, values, and the raw source line. False
positives (e.g. a table caption like "Table 2 0.95 1.2" with no real
physical meaning) are easy to spot and reject before pasting accepted rows
into PAPER_DB_MANUAL in Cell 6. The plausibility gate
(0–12 eV, 0–20 μB) already filters out most obvious noise.
Paper DB precedence rule
When the same compound exists in both Materials Project and the paper database, MP-DFT values are always kept — they come from a consistent, reproducible computational setting. Paper values only fill gaps where MP has no entry for that stoichiometry/structure combination, typically rare polymorphs or compositions outside MP's calculated set.
4. ML Architecture v6.0 — Domain-Split Ensembles + Ordering-Aware Magnetisation
The single biggest accuracy fix in v6.0 addresses a v4.0 blind spot: oxide insulators and chalcogenide insulators have systematically different gap magnitudes for the same U/W ratio, because oxygen's much higher electronegativity shifts the charge-transfer energy. v6.0 now trains separate gap models for oxides and non-oxides, plus a combined "all" model, and blends all available predictions for the new compound's chemistry.
| Stage | Model | v4.0 | v6.0 |
|---|---|---|---|
| Metal/insulator gate | RandomForestClassifier | Single model | Same, but on 27 features |
| Band gap regression | RF (+ HGB + XGBoost ensemble) | Single RF on log1p(gap) | 3 separate ensembles: oxide-only, non-oxide-only, combined — blended at inference |
| Gap calibration | Isotonic regression | None | New — corrects systematic RF bias on raw gap predictions |
| Magnetic on/off | RF classifier, balanced class weights | Implicit in mag regression | Explicit binary gate at 0.45 probability threshold |
| Magnetisation magnitude | RF + XGBoost + HGB | Features only | Features + ordering_label as an extra input column — fixes the v4.0 bug where NM/AFM/FM information was computed but never fed to the magnitude model |
| Magnetic ordering (NM/AFM/FM) | RandomForestClassifier, RandomizedSearchCV | Present | Trained first; its prediction now feeds forward into magnitude |
| Uncertainty | Bootstrap resampling | None | New — 50 bootstrap RF refits → 90% CI on every gap and magnetisation prediction |
Two compounds with identical composition-based features can have very
different magnetic moments depending on whether they order ferro- or
antiferromagnetically — FM compounds typically show larger net moments
per formula unit than AFM compounds with the same local spin. v4.0
computed the ordering classifier but never passed its output into the
magnitude regressor, silently discarding useful information. v6.0 fixes
this by appending ordering_label as a 28th input column
specifically for the magnetisation model.
Bootstrap confidence interval — code
n_bootstrap, ci_alpha = 50, 0.90
alpha = (1 - ci_alpha) / 2 # 0.05 → 5th and 95th percentile
gap_boot = np.zeros((n_bootstrap, n_test))
for b in range(n_bootstrap):
idx_b = np.random.choice(n_tr, n_tr, replace=True)
rf_b = RandomForestRegressor(n_estimators=300, random_state=b).fit(X_tr[idx_b], g_tr_log[idx_b])
gap_boot[b] = np.expm1(rf_b.predict(X_te))
gap_lo = np.quantile(gap_boot, alpha, axis=0)
gap_hi = np.quantile(gap_boot, 1-alpha, axis=0)
# Coverage check — what fraction of TRUE test values fall inside the CI?
gap_coverage = np.mean((g_te >= gap_lo) & (g_te <= gap_hi))
print(f"90% CI coverage — gap: {gap_coverage:.2%}") # target: close to 90%
A 90% confidence interval is only meaningful if roughly 90% of true test values actually fall inside it. If coverage comes out at 60%, the bootstrap is underestimating uncertainty (CIs too narrow); if it's 99%, the CIs are too wide to be useful. v6.0 prints this coverage diagnostic automatically — check it every time you retrain.
5. v4.0 → v6.0 at a Glance
| Aspect | v4.0 | v6.0 |
|---|---|---|
| Metals covered | 6 (3d only: Ti–Ni) | 23 (3d + 4d + 5d) |
| Chalcogens | S, Se, Te | O, S, Se, Te |
| Features | ~12 | 27 |
| Gap model | Single RF | 3 RF+HGB+XGB ensembles (oxide/non-oxide/all) + isotonic calibration |
| Magnetisation | Features only | Features + ordering label as 28th input |
| Uncertainty | None | 50-resample bootstrap, 90% CI with coverage check |
| Literature ingestion | Manual CSV/dict entry | Automated PDF directory scan + regex extraction (Cell 6b) |
| Hull cutoff | 0.15 eV/atom | 0.20 eV/atom (more polymorphs, still near-stable) |
| Train/test split | 80/20 random | Stratified by ordering (StratifiedShuffleSplit) |
| Hyperparameter search | Fixed parameters | RandomizedSearchCV on every sub-model |
My name is Dr Abderrahmane Reggad. I'm a 54 year old and I am a researcher in materials' sciences.
0 Comments