2601-2650 of 10000 results (33ms)
2020-12-02 ยง
21:20 <otto@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'eventstreams' for release 'canary' . [production]
21:20 <otto@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'eventstreams' for release 'production' . [production]
21:17 <otto@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'eventstreams' for release 'production' . [production]
21:16 <clarakosi@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
21:14 <clarakosi@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
21:11 <clarakosi@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
21:03 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
21:01 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:56 <twentyafterfour> deploying backports for 1.36.0-wmf.20 refs T263186 [production]
20:52 <pt1979@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:50 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
20:49 <otto@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'canary' . [production]
20:49 <otto@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'production' . [production]
20:42 <otto@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'production' . [production]
20:41 <otto@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'canary' . [production]
20:38 <otto@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'production' . [production]
20:32 <mbsantos@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'nontls' . [production]
20:32 <mbsantos@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . [production]
20:27 <mbsantos@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'nontls' . [production]
20:27 <mbsantos@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . [production]
20:25 <mbsantos@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . [production]
20:22 <mbsantos@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
20:21 <mbsantos@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
20:20 <mbsantos@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . [production]
20:14 <mutante> sodium - started update-ubuntu-mirror systemd timer - debugging why it fails; manually syncing with sudo -u mirror /usr/local/sbin/update-ubuntu-mirror [production]
20:10 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:08 <mutante> sodium systemctl reset-failed [production]
20:08 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
19:56 <mutante> sodium - systemctl restart update-tails-mirror.timer [production]
19:20 <mforns> restarted turnilo to clear deleted datasource [production]
19:17 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:643230|Add EventStream config for link recommendations (T261407)]] (duration: 01m 06s) [production]
18:22 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
18:20 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
18:20 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
18:20 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
18:00 <ladsgroup@deploy1001> Synchronized php-1.36.0-wmf.20/extensions/UrlShortener/includes/UrlShortenerUtils.php: [[gerrit:644879|Remove var_dump() left by mistake (duration: 01m 09s) [production]
17:58 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 24da542256f7c4cc955365ccd9739354f7162cc5: Add all subdomains of artsdatabanken.no to the wgCopyUploadsDomains allowlist for commonswiki (T267784) (duration: 01m 06s) [production]
17:53 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
17:53 <mutante> sodium - commenting "sync ubuntu mirror / sync tails mirror" cronjobs in the crontab of user 'mirror' after they were replaced by systemd timers by gerrit:636082 [production]
17:51 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
17:25 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=maps,service=kartotherian,name=maps1010.eqiad.wmnet [production]
17:11 <effie> uploading scap 3.16.0-1 [production]
16:25 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
16:23 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:29 <moritzm> installing libproxy security updates on Buster [production]
15:27 <moritzm> restarting turnilo [production]
15:00 <hashar@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.20 [production]
14:56 <hashar> Promoting group0 to 1.36.0-wmf.20 since I haven't done so yesterday :-\ # T263186 [production]
14:48 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Event Platform: Rename mw_session_tick stream to mediawiki.client.session_tick (duration: 01m 07s) [production]
14:19 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]