451-500 of 10000 results (25ms)
2025-07-21 §
15:48 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/rdf_subgraphs/snapshot=20250714/wiki=wikidata/scope=wikidata_main/ using stat1009.eqiad.wmnet) [production]
14:42 <bking@cumin2002> START - Cookbook sre.wdqs.data-reload reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/rdf_subgraphs/snapshot=20250714/wiki=wikidata/scope=wikidata_main/ using stat1009.eqiad.wmnet) [production]
14:42 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/rdf_subgraphs/snapshot=20250714/wiki=wikidata/scope=wikidata_main/ using stat1009.eqiad.wmnet) [production]
13:53 <bking@cumin2002> START - Cookbook sre.wdqs.data-reload reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/rdf_subgraphs/snapshot=20250714/wiki=wikidata/scope=wikidata_main/ using stat1009.eqiad.wmnet) [production]
13:53 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/rdf_subgraphs/snapshot=20250714/wiki=wikidata/ using stat1009.eqiad.wmnet) [production]
13:50 <bking@cumin2002> START - Cookbook sre.wdqs.data-reload reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/rdf_subgraphs/snapshot=20250714/wiki=wikidata/ using stat1009.eqiad.wmnet) [production]
13:49 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs://wmf/data/discovery/wikidata/rdf_subgraphs/snapshot=20250714/wiki=wikidata/ using stat1009.eqiad.wmnet) [production]
13:49 <bking@cumin2002> START - Cookbook sre.wdqs.data-reload reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs://wmf/data/discovery/wikidata/rdf_subgraphs/snapshot=20250714/wiki=wikidata/ using stat1009.eqiad.wmnet) [production]
13:48 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs://wmf/data/discovery/wikidata/rdf_subgraphs/snapshot=20250714/wiki=wikidata/ using stat1009.eqiad.wmnet) [production]
13:48 <bking@cumin2002> START - Cookbook sre.wdqs.data-reload reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs://wmf/data/discovery/wikidata/rdf_subgraphs/snapshot=20250714/wiki=wikidata/ using stat1009.eqiad.wmnet) [production]
2025-07-18 §
14:30 <marostegui@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db2242 gradually with 4 steps - Upgrade of db2242.codfw.wmnet completed [production]
13:45 <marostegui@cumin1002> START - Cookbook sre.mysql.pool db2242 gradually with 4 steps - Upgrade of db2242.codfw.wmnet completed [production]
2025-07-17 §
18:12 <dancy@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.45.0-wmf.10 refs T392180 [production]
2025-07-16 §
18:58 <swfrench-wmf> updated all shellbox instances to 2025-07-15-174312 images [production]
18:21 <dancy@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.45.0-wmf.10 refs T392180 [production]
15:02 <inflatador> bking@apt1002 publish wmf-opensearch-search-plugins_1.3.20+8_amd64 to component/opensearch13 bullseye-wikimedia [production]
08:23 <jynus@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on backup1007.eqiad.wmnet with reason: Stop minio [production]
2025-07-15 §
22:27 <swfrench-wmf> reprepro include php-excimer_1.2.5-1+wmf11u1 php-imagick_3.7.0-13+wmf11u1 php-luasandbox_4.1.2-1+wmf11u1 php-memcached_3.3.0-1+wmf11u1 php-pcov_1.0.12-1+wmf11u1 php-redis_6.2.0-1+wmf11u1 php-uuid_1.3.0-1+wmf11u1 php-wmerrors_2.0.0-1+wmf11u1 php-yaml_2.2.4-1+wmf11u1 wikidiff2_1.14.1-2+wmf11u1 xdebug_3.4.4-1+wmf11u1 in component/php83 - T398245 [production]
18:19 <dancy@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.45.0-wmf.10 refs T392180 [production]
15:00 <jynus@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db[2160,2234].codfw.wmnet,db[1217,1250].eqiad.wmnet with reason: Phorge upgrade [production]
14:22 <swfrench-wmf> reprepro include php8.1_8.1.33-1+wmf11u1 in component/php81 [production]
14:06 <swfrench-wmf> reprepro include php8.3_8.3.23-1+wmf11u2 in component/php83 - T398245 [production]
09:09 <marostegui@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db1185 gradually with 4 steps - Pool db1185.eqiad.wmnet in after cloning [production]
08:23 <marostegui@cumin1002> START - Cookbook sre.mysql.pool db1185 gradually with 4 steps - Pool db1185.eqiad.wmnet in after cloning [production]
04:01 <mwpresync@deploy1003> Pruned MediaWiki: 1.45.0-wmf.7 (duration: 01m 42s) [production]
03:48 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.45.0-wmf.10 refs T392180 (duration: 45m 36s) [production]
03:03 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.45.0-wmf.10 refs T392180 [production]
2025-07-14 §
17:59 <eevans@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on aqs1012.eqiad.wmnet with reason: Drive replacement [production]
15:46 <fceratto@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db1259 gradually with 4 steps - Pooling in [production]
15:15 <fceratto@cumin1002> START - Cookbook sre.mysql.pool db1259 gradually with 4 steps - Pooling in [production]
15:15 <fceratto@cumin1002> END (ERROR) - Cookbook sre.mysql.pool (exit_code=97) db1259 gradually with 4 steps - Pooling in [production]
15:02 <fceratto@cumin1002> START - Cookbook sre.mysql.pool db1259 gradually with 4 steps - Pooling in [production]
14:32 <fceratto@cumin1002> END (ERROR) - Cookbook sre.mysql.pool (exit_code=97) db1259 gradually with 4 steps - Pooling in [production]
14:31 <fceratto@cumin1002> START - Cookbook sre.mysql.pool db1259 gradually with 4 steps - Pooling in [production]
14:31 <fceratto@cumin1002> END (ERROR) - Cookbook sre.mysql.pool (exit_code=97) db1259 gradually with 4 steps - Pooling in [production]
13:54 <fceratto@cumin1002> START - Cookbook sre.mysql.pool db1259 gradually with 4 steps - Pooling in [production]
13:17 <fceratto@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db1233 gradually with 4 steps - Pool db1233.eqiad.wmnet in after cloning [production]
12:40 <jynus@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2200.codfw.wmnet with reason: MariaDB package upgrade [production]
12:31 <fceratto@cumin1002> START - Cookbook sre.mysql.pool db1233 gradually with 4 steps - Pool db1233.eqiad.wmnet in after cloning [production]
09:36 <marostegui@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db1200 gradually with 4 steps - Pool db1200.eqiad.wmnet in after cloning [production]
08:51 <marostegui@cumin1002> START - Cookbook sre.mysql.pool db1200 gradually with 4 steps - Pool db1200.eqiad.wmnet in after cloning [production]
2025-07-11 §
13:03 <fceratto@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) es1034 gradually with 4 steps - Pooling in [production]
12:20 <fceratto@cumin1002> START - Cookbook sre.mysql.pool es1034 gradually with 4 steps - Pooling in [production]
12:20 <fceratto@cumin1002> END (FAIL) - Cookbook sre.mysql.pool (exit_code=99) es1034 gradually with 4 steps - Pooling in [production]
12:20 <fceratto@cumin1002> START - Cookbook sre.mysql.pool es1034 gradually with 4 steps - Pooling in [production]
12:20 <fceratto@cumin1002> END (FAIL) - Cookbook sre.mysql.pool (exit_code=99) es1034 gradually with 4 steps - Pooling in [production]
12:20 <fceratto@cumin1002> START - Cookbook sre.mysql.pool es1034 gradually with 4 steps - Pooling in [production]
2025-07-10 §
09:15 <fceratto@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db2240 gradually with 4 steps - Pooling in [production]
09:15 <fceratto@cumin1002> START - Cookbook sre.mysql.pool db2240 gradually with 4 steps - Pooling in [production]
09:14 <fceratto@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db2161 gradually with 4 steps - Pooling in [production]