3351-3400 of 10000 results (34ms)
2024-08-21 §
16:45 <swfrench-wmf> imported php-pcov_1.0.11-5+wmf11u1 into component/php81 - T372507 [production]
16:45 <swfrench-wmf> imported php-msgpack_2.2.0-4+wmf11u1 into component/php81 - T372507 [production]
16:44 <swfrench-wmf> imported php-luasandbox_4.1.2-1+wmf11u1 into component/php81 - T372507 [production]
16:42 <swfrench-wmf> imported php-imagick_3.7.0-6+wmf11u1 into component/php81 - T372507 [production]
16:41 <swfrench-wmf> imported php-excimer_1.2.2-1+wmf11u1 into component/php81 - T372507 [production]
16:40 <swfrench-wmf> imported php-apcu_5.1.23-1+wmf11u1 into component/php81 - T372507 [production]
08:15 <aklapper@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.43.0-wmf.19 refs T366964 [production]
2024-08-20 §
21:04 <swfrench-wmf> imported dh-php_5.4+wmf11u1 into component/php81 - T372507 [production]
20:53 <swfrench-wmf> imported php-defaults_92+wmf11u1 into component/php81 - T372507 [production]
20:31 <swfrench-wmf> imported php8.1_8.1.29-1+wmf11u1 into component/php81 - T372507 [production]
17:51 <swfrench-wmf> mediawiki statsd exporter deployments upgraded to bookworm-based image - T368366 [production]
16:28 <mutante> LDAP - removed htriedman from wmf group, added htriedman to nda group (T371644) [production]
08:15 <aklapper@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.19 refs T366964 [production]
04:00 <mwpresync@deploy1003> Pruned MediaWiki: 1.43.0-wmf.16 (duration: 00m 56s) [production]
03:48 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.43.0-wmf.19 refs T366964 (duration: 46m 32s) [production]
03:02 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.19 refs T366964 [production]
2024-08-19 §
17:29 <swfrench-wmf> statsd-exporter resource bumps (https://gerrit.wikimedia.org/r/1061856) are now everywhere - T371885 [production]
2024-08-15 §
18:15 <jhuneidi@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.43.0-wmf.18 refs T366963 [production]
2024-08-14 §
18:14 <jhuneidi@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.43.0-wmf.18 refs T366963 [production]
14:58 <arnaudb@cumin1002> dbctl commit (dc=all): 'es1029 (re)pooling @ 4%: broken disk replaced, slow repooling', diff saved to https://phabricator.wikimedia.org/P67307 and previous config saved to /var/cache/conftool/dbconfig/20240814-145819-arnaudb.json [production]
11:52 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2189 (re)pooling @ 4%: corrupted index fixed', diff saved to https://phabricator.wikimedia.org/P67289 and previous config saved to /var/cache/conftool/dbconfig/20240814-115223-arnaudb.json [production]
09:11 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2189.codfw.wmnet with reason: replication still catching up [production]
09:11 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2189.codfw.wmnet with reason: replication still catching up [production]
07:45 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2189.codfw.wmnet with reason: index corruption [production]
07:45 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2189.codfw.wmnet with reason: index corruption [production]
2024-08-13 §
19:05 <jhuneidi@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.18 refs T366963 [production]
17:26 <swfrench-wmf> run-puppet-agent on deploy1003 to pick up scap.cfg change for T371904 [production]
04:00 <mwpresync@deploy1003> Pruned MediaWiki: 1.43.0-wmf.15 (duration: 00m 56s) [production]
03:50 <mwpresync@deploy1003> Finished scap: testwikis to 1.43.0-wmf.18 refs T366963 (duration: 48m 26s) [production]
03:02 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.18 refs T366963 [production]
2024-08-11 §
03:39 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-reload (exit_code=0) reloading scholarly_articles on wdqs1023.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/scholarly/20240729/ using stat1009.eqiad.wmnet) [production]
2024-08-09 §
05:24 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-reload (exit_code=0) reloading wikidata_main on wdqs1021.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20240729/ using stat1009.eqiad.wmnet) [production]
2024-08-08 §
17:11 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-reload reloading scholarly_articles on wdqs1023.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/scholarly/20240729/ using stat1009.eqiad.wmnet) [production]
15:21 <klausman@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on ml-serve2004.codfw.wmnet with reason: Hardware maintenance for memory errors [production]
15:21 <klausman@cumin2002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on ml-serve2004.codfw.wmnet with reason: Hardware maintenance for memory errors [production]
12:47 <jnuche@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.43.0-wmf.17 refs T366962 [production]
10:39 <jnuche@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.43.0-wmf.17 refs T366962 [production]
09:53 <jnuche@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.43.0-wmf.17 refs T366962 [production]
2024-08-07 §
08:18 <jnuche@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.43.0-wmf.17 refs T366962 [production]
2024-08-06 §
20:26 <ryankemper@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) reloading scholarly_articles on wdqs1023.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20240729/ using stat1009.eqiad.wmnet) [production]
19:49 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-reload reloading scholarly_articles on wdqs1023.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20240729/ using stat1009.eqiad.wmnet) [production]
08:16 <jnuche@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.17 refs T366962 [production]
05:41 <ryankemper@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) reloading scholarly_articles on wdqs1023.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20240729/ using stat1009.eqiad.wmnet) [production]
04:39 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-reload reloading scholarly_articles on wdqs1023.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20240729/ using stat1009.eqiad.wmnet) [production]
04:38 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-reload reloading wikidata_main on wdqs1021.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20240729/ using stat1009.eqiad.wmnet) [production]
04:01 <mwpresync@deploy1003> Pruned MediaWiki: 1.43.0-wmf.14 (duration: 00m 58s) [production]
03:47 <mwpresync@deploy1003> Finished scap: testwikis to 1.43.0-wmf.17 refs T366962 (duration: 45m 05s) [production]
03:02 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.17 refs T366962 [production]
2024-08-05 §
19:29 <ryankemper@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) reloading wikidata_main on wdqs1021.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20240729/ using stat1009.eqiad.wmnet) [production]
18:52 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-reload reloading wikidata_main on wdqs1021.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20240729/ using stat1009.eqiad.wmnet) [production]