51-100 of 10000 results (22ms)
2021-05-13 §
18:25 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission for hosts people2001.codfw.wmnet [production]
18:22 <Urbanecm> Start server-side upload for 2 video files (T282643, T282644) [production]
18:21 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 4cd6a782a946e121f5f8301e2649be8d338baaf8: Growth features: Push elwiki and cawiki out of dark mode (T280673; T280172) (duration: 01m 07s) [production]
18:19 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 04eb9d30b069e60004a42fcb128a958a24aee229: Enable media change tags on wikipedias (T266067) (duration: 01m 07s) [production]
18:09 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: b3300c3: 59c8448: Enable Extension:MediaSearch on (test)commons (T265939) (duration: 01m 08s) [production]
17:20 <andrew@deploy1002> Finished deploy [horizon/deploy@3d160f6]: Adding Database dashboards (duration: 04m 08s) [production]
17:16 <andrew@deploy1002> Started deploy [horizon/deploy@3d160f6]: Adding Database dashboards [production]
16:36 <jiji@deploy1002> Synchronized wmf-config/ProductionServices.php: Config: [[gerrit:690558|ProductionServices: add poolcounter1005 back to config (T273278)]] (duration: 01m 07s) [production]
16:26 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host poolcounter1005.eqiad.wmnet [production]
16:24 <jiji@cumin1001> START - Cookbook sre.hosts.reboot-single for host poolcounter1005.eqiad.wmnet [production]
16:24 <effie> rebooting poolcounter1005 [production]
16:09 <jiji@deploy1002> Synchronized wmf-config/ProductionServices.php: Config: [[gerrit:690557|ProductionServices: poolcounter1005 will be rebooted for updates (T273278)]] (duration: 01m 07s) [production]
15:58 <jiji@deploy1002> Synchronized wmf-config/ProductionServices.php: Config: [[gerrit:690530|ProductionServices: add poolcounter1004 back to config (T273278)]] (duration: 01m 07s) [production]
15:49 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host poolcounter1004.eqiad.wmnet [production]
15:46 <jiji@cumin1001> START - Cookbook sre.hosts.reboot-single for host poolcounter1004.eqiad.wmnet [production]
15:46 <effie> restarting poolcounter1004 [production]
15:27 <jiji@deploy1002> Synchronized wmf-config/ProductionServices.php: Config: [[gerrit:688239|ProductionServices: poolcounter1004 will be rebooted for updates (T273278)]] (duration: 01m 08s) [production]
14:49 <Urbanecm> Start server-side upload for 1 video file (T282785) [production]
14:07 <Urbanecm> Start server-side upload for 3 video files (T282558, T282556) [production]
12:40 <tgr@deploy1002> Synchronized php-1.37.0-wmf.5/extensions/GrowthExperiments: Backport: instrumentation patches ([[gerrit:690070|]] [[gerrit:690071|]] [[gerrit:690072|]] [[gerrit:690073|]]) (T278116 T278117 T278114 T278177 T278487 T278112 T278111 T278118) (duration: 01m 09s) [production]
11:00 <hnowlan> deleting packages still referenced by jessie components: `sudo -i reprepro clearvanished --delete` [production]
10:46 <mvolz@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'citoid' for release 'production' . [production]
10:40 <mvolz@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'citoid' for release 'staging' . [production]
10:31 <mvolz@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'zotero' for release 'production' . [production]
10:25 <mvolz@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'zotero' for release 'production' . [production]
10:11 <mvolz@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'zotero' for release 'staging' . [production]
08:47 <akosiaris@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'internal' . [production]
08:47 <akosiaris@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . [production]
08:45 <akosiaris@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . [production]
08:45 <akosiaris@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'internal' . [production]
08:21 <akosiaris@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'linkrecommendation' for release 'staging' . [production]
07:43 <kevinbazira@deploy1002> Finished deploy [ores/deploy@8fd23ed]: Regular ORES Deployment T278723 (duration: 32m 50s) [production]
07:10 <kevinbazira@deploy1002> Started deploy [ores/deploy@8fd23ed]: Regular ORES Deployment T278723 [production]
05:54 <_joe_> running docker image prune on contint1001, which has 722 unlinked images stored in its docker daemon [production]
01:20 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
2021-05-12 §
23:48 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.4/extensions/WikiEditor/includes/WikiEditorHooks.php: 2f6af514c49d47bbec5ce51f9f7263015e039003? PHP VisualEditorFeatureUse logging: properly record session id (T281409) (duration: 01m 07s) [production]
23:40 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.5/extensions/WikiEditor/includes/WikiEditorHooks.php: ef4139628a36eb8b747c610c8d769a802faf2fc3: PHP VisualEditorFeatureUse logging: properly record session id (T281409) (duration: 01m 08s) [production]
23:27 <ryankemper> T280382 `sudo -i cookbook sre.wdqs.data-transfer --source wdqs2001.codfw.wmnet --dest wdqs2007.codfw.wmnet --reason "transferring fresh wikidata journal following reimage" --blazegraph_instance blazegraph` on `ryankemper@cumin2001` tmux session `wdqs_reimage` [production]
23:27 <ryankemper> T280382 `sudo -i cookbook sre.wdqs.data-transfer --source wdqs2001.codfw.wmnet --dest wdqs2007.codfw.wmnet --reason "transferring fresh wikidata journal following reimage" --blazegraph_instance blazegraph` on `ryankemper@cumin1001` tmux session `reimage` [production]
23:27 <ryankemper@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
22:01 <ryankemper@cumin2001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
21:56 <ryankemper> T280382 `sudo -i cookbook sre.wdqs.data-transfer --source wdqs2001.codfw.wmnet --dest wdqs2007.codfw.wmnet --reason "transferring fresh categories journal following reimage" --blazegraph_instance categories` on `ryankemper@cumin1001` tmux session `reimage` [production]
21:56 <ryankemper@cumin2001> START - Cookbook sre.wdqs.data-transfer [production]
21:54 <ryankemper> T280382 `wdqs1012.eqiad.wmnet` has been re-imaged and had the appropriate wikidata/categories journal files transferred. `df -h` shows disk space is no longer an issue following the switch to `raid0`: `/dev/mapper/vg0-srv 2.7T 998G 1.6T 39% /srv` [production]
20:57 <ottomata> starting new drop_event data purge job to drop all event data older than 90 days in the Hive event database - T273789 [production]
20:33 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
19:27 <ryankemper@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs2007.codfw.wmnet with reason: REIMAGE [production]
19:25 <ryankemper@cumin2001> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs2007.codfw.wmnet with reason: REIMAGE [production]
19:15 <ryankemper> T280382 `sudo -i cookbook sre.wdqs.data-transfer --source wdqs1011.eqiad.wmnet --dest wdqs1012.eqiad.wmnet --reason "transferring fresh wikidata journal following reimage" --blazegraph_instance blazegraph` on `ryankemper@cumin1001` tmux session `wdqs_reimage` [production]
19:15 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]