All pull requests

Pull requests

• fix(benchmarking): propagate entry environment to Ray startup

  #2548·rlratzel opened Oct 30, 2026Contributor·

  ·
  10

• refactor: Rename Indic Canary to Indic Transcribe model

  #2547·sushmitha-deva-09 opened Oct 30, 2026Contributor·3 tasks·

  ·
  8

• fix(text): make RepeatingTopNGramsFilter tie-break deterministic

  community-request
  #2544·pujitha24 opened Oct 29, 2026·1 of 3 tasks·

  ·
  1

  3

• fix(text): normalize PerExtensionFilter language and extension keys

  community-request
  #2543·kjgggggg opened Oct 29, 2026·2 of 3 tasks·

  ·
  2

• fix(arxiv): expand non-arg macros literally so non-ASCII values work

  community-request
  #2542·kjgggggg opened Oct 29, 2026·2 of 3 tasks·

  ·
  2

• fix(text): match CJK n-grams without spaces in repetition filters

  community-request
  #2541·kjgggggg opened Oct 29, 2026·2 of 3 tasks·

  ·
  3

• fix(audio): enable ARM dependencies and validate runtime setup

  #2538·rlratzel opened Oct 26, 2026Contributor·

  ·
  27

• feat(audio): add sharded Curator MFA alignment with VAD refinement

  #2537·timofeeva16tanya opened Oct 26, 2026·

  ·
  17

• feat(image): add reusable scoring and selective diagnostics stages

  #2536·VibhuJawa opened Oct 25, 2026Contributor·

  ·
  1

• fix(arxiv): keep the character before an inline comment

  community-requestwaiting-on-maintainers
  #2534·alphasafal opened Oct 25, 2026·3 tasks done·

  ·
  3

• docs: fix typos in arXiv extractor and Slicer comments

  community-requestwaiting-on-maintainers
  #2533·Avicennasis opened Oct 25, 2026·

  ·
  2

• perf(interleaved): streamline single-content WebDataset writes

  #2532·VibhuJawa opened Oct 25, 2026Contributor·

  ·
  2

• fix(interleaved): omit range offsets for compressed WebDataset shards

  #2531·VibhuJawa opened Oct 24, 2026Contributor·

  ·
  2

• feat(pdf): support managed Nemotron Parse 3.5 on CUDA 13

  community-requestwaiting-on-maintainers
  #2530·jperez999 opened Oct 23, 2026·

  ·
  13

• fix(string): SymbolsToWordsFilter counts individual '' characters, not runs

  community-requestwaiting-on-maintainers
  #2528·cf3901646 opened Oct 22, 2026·

  ·
  1

  4

• feat(interleaved): extract image references from document batches and MinerU

  #2527·VibhuJawa opened Oct 22, 2026Contributor·3 tasks done·

  ·
  1

• feat(interleaved): support served Nemotron Parse 3.5 modes

  community-requestwaiting-on-maintainers
  #2526·KyleZheng1284 opened Oct 22, 2026·3 tasks done·

  ·
  5

• fix: bound inference startup and repair OCR installation

  #2525·praateekmahajan opened Oct 22, 2026Contributor·3 tasks done·

  ·
  1

• feat(audio): bundle original FastMSS for meeting simulation

  #2524·timofeeva16tanya opened Oct 22, 2026·4 of 6 tasks·

  ·
  8

• feat(dedup): upgrade RAPIDS to 26.10 and use ParquetDatasetWriter

  #2523·praateekmahajan opened Oct 22, 2026Contributor·

  ·
  4

• fix(text): keep punctuation that follows a URL in UrlRemover

  community-requestwaiting-on-maintainers
  #2520·usmanmateen opened Oct 20, 2026Contributor·3 tasks done·

  ·
  1

  6

• fix(text): count hash characters, not runs, in SymbolsToWordsFilter

  community-requestwaiting-on-maintainers
  #2519·usmanmateen opened Oct 20, 2026Contributor·3 tasks done·

  ·
  1

  2

• fix(dedup): explain how to start Ray when the ID generator actor is missing

  community-requestwaiting-on-maintainers
  #2514·hzmsh opened Oct 19, 2026·3 tasks done·

  ·
  1

  3

• fix(text): compute histogram filter ratio over stripped text

  community-requestwaiting-on-maintainers
  #2511·udsy19 opened Oct 17, 2026·

  ·
  3

• fix(utils): match multi-part file extensions such as .jsonl.gz

  community-requestwaiting-on-maintainers
  #2510·hzmsh opened Oct 16, 2026·3 tasks done·

  ·
  1

  2