1-50 of 10000 results (46ms)
2026-09-10 ยง
19:04 <bking@cumin2003> START - Cookbook sre.hadoop.roll-restart-workers restart workers for Hadoop test cluster: Roll restart of jvm daemons for openjdk upgrade. [production]
18:21 <otto@deploy1003> Finished deploy [analytics/refinery@92b5c04] (thin): Regular analytics weekly train THIN [analytics/refinery@92b5c04e] (duration: 00m 59s) [production]
18:20 <otto@deploy1003> Started deploy [analytics/refinery@92b5c04] (thin): Regular analytics weekly train THIN [analytics/refinery@92b5c04e] [production]
18:19 <otto@deploy1003> Finished deploy [analytics/refinery@92b5c04]: Regular analytics weekly train [analytics/refinery@92b5c04e] (duration: 05m 13s) [production]
18:14 <btullis@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/services/mediawiki-dumps-legacy: apply [production]
18:14 <otto@deploy1003> Started deploy [analytics/refinery@92b5c04]: Regular analytics weekly train [analytics/refinery@92b5c04e] [production]
18:13 <otto@deploy1003> Finished deploy [analytics/refinery@92b5c04] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@92b5c04e] (duration: 00m 39s) [production]
18:13 <otto@deploy1003> Started deploy [analytics/refinery@92b5c04] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@92b5c04e] [production]
18:13 <dbrant@deploy1003> helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply [production]
18:12 <dbrant@deploy1003> helmfile [codfw] START helmfile.d/services/wikifeeds: apply [production]
18:11 <dbrant@deploy1003> helmfile [eqiad] DONE helmfile.d/services/wikifeeds: apply [production]
18:11 <dzahn@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on gerrit2002.wikimedia.org with reason: maintenance upgrade [production]
18:11 <dancy@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.47.0-wmf.19 refs T430838 [production]
18:11 <dbrant@deploy1003> helmfile [eqiad] START helmfile.d/services/wikifeeds: apply [production]
18:10 <dzahn@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on gerrit1003.wikimedia.org with reason: maintenance upgrade [production]
18:09 <dbrant@deploy1003> helmfile [staging] DONE helmfile.d/services/wikifeeds: apply [production]
18:08 <dbrant@deploy1003> helmfile [staging] START helmfile.d/services/wikifeeds: apply [production]
18:06 <btullis@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/services/mediawiki-dumps-legacy: apply [production]
16:40 <urbanecm@deploy1003> Finished scap sync-world: Backport for [[gerrit:1338984|refactor: Provider: Add IConfigurationProvider::invalidateCache() (T437588)]], [[gerrit:1338990|Provider: Cache the valid configuration in the process (T437588)]], [[gerrit:1338983|refactor: Provider: Add IConfigurationProvider::invalidateCache() (T437588)]], [[gerrit:1338986|Provider: Cache the valid configuration in the process (T437588)]] [production]
16:35 <urbanecm@deploy1003> urbanecm: Continuing with deployment [production]
16:33 <urbanecm@deploy1003> urbanecm: Backport for [[gerrit:1338984|refactor: Provider: Add IConfigurationProvider::invalidateCache() (T437588)]], [[gerrit:1338990|Provider: Cache the valid configuration in the process (T437588)]], [[gerrit:1338983|refactor: Provider: Add IConfigurationProvider::invalidateCache() (T437588)]], [[gerrit:1338986|Provider: Cache the valid configuration in the process (T437588)]] synced to the te [production]
16:28 <urbanecm@deploy1003> Started scap sync-world: Backport for [[gerrit:1338984|refactor: Provider: Add IConfigurationProvider::invalidateCache() (T437588)]], [[gerrit:1338990|Provider: Cache the valid configuration in the process (T437588)]], [[gerrit:1338983|refactor: Provider: Add IConfigurationProvider::invalidateCache() (T437588)]], [[gerrit:1338986|Provider: Cache the valid configuration in the process (T437588)]] [production]
15:33 <marostegui@cumin1003> conftool action : set/pooled=no; selector: name=clouddb1025.eqiad.wmnet,service=s4 [production]
15:04 <samtar@deploy1003> Finished scap sync-world: Backport for [[gerrit:1335032|IS: enable wgEnableWatchstarPopover on test.wikipedia (T436955)]] (duration: 08m 08s) [production]
15:02 <bking@cumin2003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddumps1001.wikimedia.org with OS bookworm [production]
14:59 <samtar@deploy1003> samtar: Continuing with deployment [production]
14:58 <samtar@deploy1003> samtar: Backport for [[gerrit:1335032|IS: enable wgEnableWatchstarPopover on test.wikipedia (T436955)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
14:56 <samtar@deploy1003> Started scap sync-world: Backport for [[gerrit:1335032|IS: enable wgEnableWatchstarPopover on test.wikipedia (T436955)]] [production]
14:40 <bking@cumin2003> END (PASS) - Cookbook sre.presto.roll-restart-workers (exit_code=0) for Presto an-presto cluster: Roll restart of all Presto's jvm daemons. [production]
14:21 <cdanis@cumin1004> END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "etcd fanout fix & details UX - cdanis@cumin1004" [production]
14:21 <cdanis@cumin1004> END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: etcd fanout fix & details UX - cdanis@cumin1004 [production]
14:20 <cdanis@cumin1004> START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: etcd fanout fix & details UX - cdanis@cumin1004 [production]
14:20 <cdanis@cumin1004> START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "etcd fanout fix & details UX - cdanis@cumin1004" [production]
14:17 <bking@cumin2003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddumps1001.wikimedia.org with reason: host reimage [production]
14:13 <bking@cumin2003> START - Cookbook sre.hosts.downtime for 2:00:00 on clouddumps1001.wikimedia.org with reason: host reimage [production]
14:07 <bking@cumin2003> START - Cookbook sre.presto.roll-restart-workers for Presto an-presto cluster: Roll restart of all Presto's jvm daemons. [production]
13:57 <bking@cumin2003> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bookworm [production]
13:56 <jelto@cumin1003> END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-eqiad@eqiad [production]
13:56 <jelto@cumin1003> END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs [production]
13:55 <jelto@cumin1003> START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-eqiad or A:lvs-secondary-eqiad) and A:bullseye and A:lvs [production]
13:51 <jelto@cumin1003> START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-eqiad@eqiad [production]
13:42 <jelto@cumin1003> END (PASS) - Cookbook sre.loadbalancer.migrate-service-ipip (exit_code=0) for alias: wikikube-worker-codfw@codfw [production]
13:41 <jelto@cumin1003> END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs [production]
13:41 <jelto@cumin1003> START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on (A:lvs-low-traffic-codfw or A:lvs-secondary-codfw) and A:bullseye and A:lvs [production]
13:37 <jelto@cumin1003> START - Cookbook sre.loadbalancer.migrate-service-ipip for alias: wikikube-worker-codfw@codfw [production]
12:48 <klausman@dns1004> END - running authdns-update [production]
12:46 <klausman@dns1004> START - running authdns-update [production]
12:35 <dbrant@deploy1003> helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply [production]
12:35 <dbrant@deploy1003> helmfile [codfw] START helmfile.d/services/wikifeeds: apply [production]
12:34 <dbrant@deploy1003> helmfile [eqiad] DONE helmfile.d/services/wikifeeds: apply [production]