851-900 of 10000 results (40ms)
2021-03-19 §
11:29 <klausman@deploy1002> helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. [production]
11:29 <klausman@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
11:29 <klausman@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
11:27 <akosiaris@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
11:27 <akosiaris@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
11:20 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ml-serve2002.codfw.wmnet with reason: REIMAGE [production]
11:18 <elukey@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on ml-serve2002.codfw.wmnet with reason: REIMAGE [production]
10:45 <klausman@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
10:45 <klausman@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
10:42 <moritzm> installing dbmonitor1002 T224589 [production]
10:42 <klausman@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
10:42 <klausman@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
10:41 <klausman@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
10:41 <klausman@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
10:11 <kharlan@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . [production]
10:10 <kharlan@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . [production]
10:05 <kharlan@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . [production]
10:04 <kharlan@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . [production]
09:40 <kharlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'linkrecommendation' for release 'staging' . [production]
09:36 <jayme@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'proton' for release 'production' . [production]
08:22 <elukey> upload alluxio 2.4.1 to thirdparty/bigtop15 on stretch/buster-wikimedia [production]
07:16 <ryankemper> T275885 `ryankemper@cumin1001:~$ sudo cumin 'P{relforge*}' 'sudo run-puppet-agent'` (change hadn't been merged when I ran the agent earlier) [production]
04:04 <eileen> civicrm revision changed from 99bf1c9210 to 39d24e8b0a, config revision is 26b02db7ba [production]
03:27 <ryankemper> [wdqs] `ryankemper@wdqs1013:~$ sudo systemctl restart wdqs-blazegraph` [production]
03:26 <ryankemper> T275885 `ryankemper@cumin1001:~$ sudo cumin 'P{relforge*}' 'sudo run-puppet-agent'` [production]
02:43 <ryankemper> T275885 Revoking current `relforge` TLS cert in advance of generation of new cert: `ryankemper@puppetmaster1001:/srv/private$ sudo puppet cert clean relforge.svc.eqiad.wmnet` [production]
00:51 <dancy@deploy1002> Synchronized php-1.36.0-wmf.35/extensions/LiquidThreads/classes/Thread.php: T277772 (duration: 00m 58s) [production]
00:45 <mutante> testreduce1001 - stop mysql; rsyncing /var/lib/mysql to /srv/data/mysql (T277580) [production]
2021-03-18 §
23:56 <legoktm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Don't define a default icon (T274199) (duration: 00m 57s) [production]
23:38 <brennen@deploy1002> Synchronized php-1.36.0-wmf.35/includes/user/ActorStore.php: Backport: [[gerrit:673115|ActorStore::getActorById - fall back to master. (T277795)]] (duration: 00m 57s) [production]
23:35 <brennen@deploy1002> Synchronized php-1.36.0-wmf.35/includes/user/ActorStore.php: Backport: [[gerrit:673115|ActorStore::getActorById - fall back to master. (T277795)]] (duration: 00m 58s) [production]
23:25 <dduvall@deploy1002> Synchronized .pipeline: config: [[gerrit:673375|Use build environment HTTP proxy for APT sources (T277109)]] (duration: 01m 02s) [production]
23:06 <brennen> train status: 1.36.0-wmf.35 (T274939) stable on all wikis after deploy of hotfix for T277795 [production]
22:53 <brennen@deploy1002> Synchronized php-1.36.0-wmf.35/includes/specials/SpecialContributions.php: Backport: [[gerrit:673115|ActorStore::getActorById - fall back to master. (T277795)]] (duration: 01m 07s) [production]
22:30 <dduvall@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'blubberoid' for release 'production' . [production]
22:29 <dduvall@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'blubberoid' for release 'production' . [production]
22:25 <dduvall@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . [production]
20:37 <dancy@deploy1002> Synchronized php-1.36.0-wmf.35/extensions/LiquidThreads/classes/Thread.php: (no justification provided) (duration: 01m 05s) [production]
19:04 <dancy@deploy1002> rebuilt and synchronized wikiversions files: group2 wikis to 1.36.0-wmf.35 [production]
18:28 <legoktm> re-enabled puppet on registry* [production]
18:17 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 44eddcc: hrwiki: Deploy Growth features to newcomers (T275684) (duration: 01m 08s) [production]
18:12 <urbanecm@deploy1002> Synchronized dblists/growthexperiments.dblist: 179d9e5: mswiki: Enable Growth features in stealth mode (T277562; 2/2) (duration: 01m 08s) [production]
18:10 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 179d9e5: mswiki: Enable Growth features in stealth mode (T277562; 1/2) (duration: 01m 11s) [production]
17:58 <legoktm> disabled puppet on registry* for rolling out https://gerrit.wikimedia.org/r/672537 [production]
17:50 <urbanecm@deploy1002> Synchronized dblists/growthexperiments.dblist: 55aa6cb: tewiki: Enable Growth features in stealth mode (T277491; 2/2) (duration: 01m 08s) [production]
17:50 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts mw2242.codfw.wmnet [production]
17:48 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 55aa6cb: tewiki: Enable Growth features in stealth mode (T277491; 1/2) (duration: 01m 10s) [production]
17:45 <urbanecm@deploy1002> Synchronized dblists/growthexperiments.dblist: 04342e9bb0765a6a58ad78bd7eaa380d4167f0c1: simplewiki: Enable Growth team features in stealth mode (T277550) (duration: 01m 09s) [production]
17:42 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 04342e9bb0765a6a58ad78bd7eaa380d4167f0c1: simplewiki: Enable Growth team features in stealth mode (T277550) (duration: 01m 10s) [production]
17:40 <dduvall@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . [production]