1801-1850 of 10000 results (22ms)
2024-08-23 §
15:17 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 14:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
15:17 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 4 days, 14:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
13:32 <jayme@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on kafka-main2001.codfw.wmnet with reason: Decom next week [production]
13:32 <jayme@cumin1002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on kafka-main2001.codfw.wmnet with reason: Decom next week [production]
2024-08-22 §
20:19 <swfrench-wmf> imported wikidiff2_1.14.1-2+wmf11u2 into component/php81 - T372507 [production]
20:18 <swfrench-wmf> imported php-wmerrors_2.0.0-1+wmf11u2 into component/php81 - T372507 [production]
20:17 <swfrench-wmf> imported php-luasandbox_4.1.2-1+wmf11u2 into component/php81 - T372507 [production]
08:16 <aklapper@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.43.0-wmf.19 refs T366964 [production]
2024-08-21 §
18:21 <swfrench-wmf> imported php-memcached_3.2.0++-1+wmf11u1 into component/php81 - T372507 [production]
18:13 <swfrench-wmf> imported php-redis_6.0.2-1+wmf11u1 into component/php81 - T372507 [production]
17:56 <swfrench-wmf> imported php-igbinary_3.2.15-1+wmf11u1 into component/php81 - T372507 [production]
17:37 <swfrench-wmf> imported xdebug_3.3.2-1+wmf11u1 into component/php81 - T372507 [production]
17:36 <swfrench-wmf> imported wikidiff2_1.14.1-2+wmf11u1 into component/php81 - T372507 [production]
17:35 <swfrench-wmf> imported tideways_5.0.4-16+wmf11u1 into component/php81 - T372507 [production]
17:34 <swfrench-wmf> imported php-yaml_2.2.3-2+wmf11u1 into component/php81 - T372507 [production]
17:34 <swfrench-wmf> imported php-wmerrors_2.0.0-1+wmf11u1 into component/php81 - T372507 [production]
16:45 <swfrench-wmf> imported php-pcov_1.0.11-5+wmf11u1 into component/php81 - T372507 [production]
16:45 <swfrench-wmf> imported php-msgpack_2.2.0-4+wmf11u1 into component/php81 - T372507 [production]
16:44 <swfrench-wmf> imported php-luasandbox_4.1.2-1+wmf11u1 into component/php81 - T372507 [production]
16:42 <swfrench-wmf> imported php-imagick_3.7.0-6+wmf11u1 into component/php81 - T372507 [production]
16:41 <swfrench-wmf> imported php-excimer_1.2.2-1+wmf11u1 into component/php81 - T372507 [production]
16:40 <swfrench-wmf> imported php-apcu_5.1.23-1+wmf11u1 into component/php81 - T372507 [production]
08:15 <aklapper@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.43.0-wmf.19 refs T366964 [production]
2024-08-20 §
21:04 <swfrench-wmf> imported dh-php_5.4+wmf11u1 into component/php81 - T372507 [production]
20:53 <swfrench-wmf> imported php-defaults_92+wmf11u1 into component/php81 - T372507 [production]
20:31 <swfrench-wmf> imported php8.1_8.1.29-1+wmf11u1 into component/php81 - T372507 [production]
17:51 <swfrench-wmf> mediawiki statsd exporter deployments upgraded to bookworm-based image - T368366 [production]
16:28 <mutante> LDAP - removed htriedman from wmf group, added htriedman to nda group (T371644) [production]
08:15 <aklapper@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.19 refs T366964 [production]
04:00 <mwpresync@deploy1003> Pruned MediaWiki: 1.43.0-wmf.16 (duration: 00m 56s) [production]
03:48 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.43.0-wmf.19 refs T366964 (duration: 46m 32s) [production]
03:02 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.19 refs T366964 [production]
2024-08-19 §
17:29 <swfrench-wmf> statsd-exporter resource bumps (https://gerrit.wikimedia.org/r/1061856) are now everywhere - T371885 [production]
2024-08-15 §
18:15 <jhuneidi@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.43.0-wmf.18 refs T366963 [production]
2024-08-14 §
18:14 <jhuneidi@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.43.0-wmf.18 refs T366963 [production]
14:58 <arnaudb@cumin1002> dbctl commit (dc=all): 'es1029 (re)pooling @ 4%: broken disk replaced, slow repooling', diff saved to https://phabricator.wikimedia.org/P67307 and previous config saved to /var/cache/conftool/dbconfig/20240814-145819-arnaudb.json [production]
11:52 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2189 (re)pooling @ 4%: corrupted index fixed', diff saved to https://phabricator.wikimedia.org/P67289 and previous config saved to /var/cache/conftool/dbconfig/20240814-115223-arnaudb.json [production]
09:11 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2189.codfw.wmnet with reason: replication still catching up [production]
09:11 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2189.codfw.wmnet with reason: replication still catching up [production]
07:45 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2189.codfw.wmnet with reason: index corruption [production]
07:45 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2189.codfw.wmnet with reason: index corruption [production]
2024-08-13 §
19:05 <jhuneidi@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.18 refs T366963 [production]
17:26 <swfrench-wmf> run-puppet-agent on deploy1003 to pick up scap.cfg change for T371904 [production]
04:00 <mwpresync@deploy1003> Pruned MediaWiki: 1.43.0-wmf.15 (duration: 00m 56s) [production]
03:50 <mwpresync@deploy1003> Finished scap: testwikis to 1.43.0-wmf.18 refs T366963 (duration: 48m 26s) [production]
03:02 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.18 refs T366963 [production]
2024-08-11 §
03:39 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-reload (exit_code=0) reloading scholarly_articles on wdqs1023.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/scholarly/20240729/ using stat1009.eqiad.wmnet) [production]
2024-08-09 §
05:24 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-reload (exit_code=0) reloading wikidata_main on wdqs1021.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20240729/ using stat1009.eqiad.wmnet) [production]
2024-08-08 §
17:11 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-reload reloading scholarly_articles on wdqs1023.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/scholarly/20240729/ using stat1009.eqiad.wmnet) [production]
15:21 <klausman@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on ml-serve2004.codfw.wmnet with reason: Hardware maintenance for memory errors [production]