1651-1700 of 10000 results (20ms)
2024-10-03 §
22:18 <brennen@deploy2002> scap failed: <CalledProcessError> Command 'sudo -u mwbuilder /srv/mwbuilder/release/make-container-image/build-images.py /srv/mediawiki-staging/scap/image-build --staging-dir /srv/mediawiki-staging --mediawiki-versions 1.43.0-wmf.25 --multiversion-image-name docker-registry.discovery.wmnet/restricted/mediawiki-multiversion --multiversion-debug-image-name docker-registry.discovery.wmnet/restricted/m [production]
22:15 <brennen@deploy2002> scap failed: <CalledProcessError> Command 'sudo -u mwbuilder /srv/mwbuilder/release/make-container-image/build-images.py /srv/mediawiki-staging/scap/image-build --staging-dir /srv/mediawiki-staging --mediawiki-versions 1.43.0-wmf.25 --multiversion-image-name docker-registry.discovery.wmnet/restricted/mediawiki-multiversion --multiversion-debug-image-name docker-registry.discovery.wmnet/restricted/m [production]
19:25 <ryankemper> [WDQS Deploy] Restarted `wdqs-updater` across all hosts, 4 hosts at a time: `sudo -E cumin -b 4 'A:wdqs-all' 'systemctl restart wdqs-updater'` [production]
08:36 <hashar@deploy2002> rebuilt and synchronized wikiversions files: group2 to 1.43.0-wmf.25 refs T375656 [production]
2024-10-02 §
18:10 <hashar@deploy2002> rebuilt and synchronized wikiversions files: group1 to 1.43.0-wmf.25 refs T375656 [production]
09:31 <hashar@deploy2002> rebuilt and synchronized wikiversions files: Revert "group1 wikis to [php-1.43.0-wmf.24]" - T375656 [production]
07:09 <vgutierrez@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on cp[3071-3072].esams.wmnet with reason: HW maintenance [production]
07:09 <vgutierrez@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on cp[3071-3072].esams.wmnet with reason: HW maintenance [production]
2024-10-01 §
20:28 <hashar> mwscript purgeList.php --wiki=tlywiki --namespace=4 # T367009 [production]
08:58 <hashar@deploy2002> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.25 refs T375656 [production]
04:01 <mwpresync@deploy2002> Pruned MediaWiki: 1.43.0-wmf.22 (duration: 00m 58s) [production]
03:51 <mwpresync@deploy2002> Finished scap sync-world: testwikis to 1.43.0-wmf.25 refs T375656 (duration: 48m 36s) [production]
03:02 <mwpresync@deploy2002> Started scap sync-world: testwikis to 1.43.0-wmf.25 refs T375656 [production]
2024-09-30 §
18:04 <sukhe@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on lvs[4008-4010].ulsfo.wmnet with reason: site is depooled, cr3-ulsfo is being replaced [production]
18:04 <sukhe@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on lvs[4008-4010].ulsfo.wmnet with reason: site is depooled, cr3-ulsfo is being replaced [production]
16:53 <swfrench-wmf> built and published new php8.1 images - T376036 [production]
16:27 <swfrench-wmf> imported php8.1_8.1.30-1+wmf11u1 into component/php81 - T376036 [production]
2024-09-26 §
23:41 <swfrench-wmf> silenced KubernetesAPILatency for 24h for resource=~"(blockaffinities|ipamblocks)" site="eqiad" (f33e2c42-d921-4686-a761-59d43ad14d84) - T369744 [production]
19:30 <brennen@deploy2002> rebuilt and synchronized wikiversions files: group2 to 1.43.0-wmf.24 refs T373643 [production]
17:08 <swfrench-wmf> cleared contents of a stale scap dsh group resource on deploy2002 - T370962 [production]
16:27 <swfrench-wmf> done with switchover day 3 deployment server switch - T370962 [production]
15:04 <swfrench-wmf> running authdns-update for deployment CNAME switch - T370962 [production]
15:01 <swfrench-wmf> starting switchover day 3 deployment server switch - T370962 [production]
2024-09-25 §
18:14 <brennen@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.43.0-wmf.24 refs T373643 [production]
18:05 <brennen> 1.43.0-wmf.24 train status: no current blockers, logs clean-ish, rolling to group1 [production]
15:57 <swfrench-wmf> switchover day 2 manual steps are complete - T370962 [production]
14:54 <swfrench-wmf> reset failed units on mwmaint1002 to unblock 01-stop-maintenance - T370962 [production]
13:50 <swfrench-wmf> kartotherian repooled in eqiad due load issues - T370962 [production]
2024-09-24 §
18:21 <brennen@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.24 refs T373643 [production]
17:09 <brennen@deploy1003> Finished scap sync-world: testwikis to 1.43.0-wmf.24 refs T373643 (duration: 27m 45s) [production]
16:41 <brennen@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.24 refs T373643 [production]
16:38 <swfrench-wmf> switched logstash.discovery.wmnet back to eqiad due to reports of stale dashboards - T370962 [production]
16:18 <swfrench-wmf> repooled swift in eqiad to potentially mitigate issues with thumbor and swift - T370962 [production]
16:06 <swfrench-wmf> repooled swift-ro in eqiad to potentially mitigate issues with thumbor - T370962 [production]
15:50 <swfrench-wmf> switchover day 1 actions are complete - T370962 [production]
15:01 <swfrench-wmf> starting switchover day 1 - T370962 [production]
14:21 <swfrench-wmf> scaled up mw-web ahead of traffic+services switchover - T370962 [production]
14:19 <swfrench-wmf> scaled up mw-api-ext ahead of traffic+services switchover - T370962 [production]
10:10 <jnuche@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.24 refs T373643 [production]
08:43 <jnuche@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.24 refs T373643 [production]
08:30 <jnuche@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.24 refs T373643 [production]
04:06 <mwpresync@deploy1003> Pruned MediaWiki: 1.43.0-wmf.21 (duration: 06m 02s) [production]
03:02 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.24 refs T373643 [production]
2024-09-23 §
09:22 <jayme@cumin1002> conftool action : set/pooled=no; selector: name=registry200(3|4).codfw.wmnet [production]
2024-09-20 §
13:09 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on mw2425.codfw.wmnet with reason: reimage [production]
13:09 <jiji@cumin1002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on mw2425.codfw.wmnet with reason: reimage [production]
2024-09-19 §
18:02 <swfrench-wmf> scaling down mw-api-ext in eqiad after pre-switchover testing - T371273 [production]
18:02 <swfrench-wmf> scaling down mw-web in eqiad after pre-switchover testing - T371273 [production]
16:50 <swfrench-wmf> scaling up mw-web in eqiad for pre-switchover testing - T371273 [production]
16:48 <swfrench-wmf> scaling up mw-api-ext in eqiad for pre-switchover testing - T371273 [production]