801-850 of 10000 results (30ms)
2020-10-08 §
07:00 <dcausse> depooling wdqs2002 (catching-up lag) [production]
06:57 <dcausse> restart blazegraph on wdqs2002 (stuck) T242453 [production]
06:51 <_joe_> enable notifications for wdqs-ssl-codfw [production]
05:33 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
05:27 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission [production]
04:05 <ejegg> updated fundraising python tools from 5515923ef7 to d4e08c52de [production]
00:31 <tgr_> evening deploys done [production]
00:20 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:632796|Enable logging of session cookie changes in group1 (T264793)]] (again, forgot to rebase the previous time) (duration: 00m 59s) [production]
00:15 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:632796|Enable logging of session cookie changes in group1 (T264793)]] (duration: 00m 57s) [production]
00:03 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:632795|Enable logging of session cookie changes in group0 (T264793)]] (duration: 00m 58s) [production]
2020-10-07 §
23:58 <tgr@deploy1001> Synchronized php-1.36.0-wmf.10/includes/session: Backport: [[gerrit:632685|Log when SessionManager is emitting cookies (T264793)]] (duration: 01m 00s) [production]
23:56 <ryankemper@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=0) [production]
23:56 <ryankemper@cumin1001> START - Cookbook sre.elasticsearch.rolling-reboot [production]
23:56 <ryankemper@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-restart (exit_code=0) [production]
23:55 <ryankemper@cumin1001> START - Cookbook sre.elasticsearch.rolling-restart [production]
21:55 <ryankemper@cumin1001> END (FAIL) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=99) [production]
21:48 <ryankemper@cumin1001> START - Cookbook sre.elasticsearch.rolling-reboot [production]
21:14 <ryankemper@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=0) [production]
20:56 <ryankemper@cumin1001> START - Cookbook sre.elasticsearch.rolling-reboot [production]
20:09 <ebernhardson@deploy1001> Finished deploy [wikimedia/discovery/analytics@7fa787e]: airflow: update mjolnir configuration to reduce max training dataset (duration: 03m 23s) [production]
20:05 <ebernhardson@deploy1001> Started deploy [wikimedia/discovery/analytics@7fa787e]: airflow: update mjolnir configuration to reduce max training dataset [production]
19:36 <mutante> blog post: The latest addition to our family of Wikimedia languages is "Inari Sami" with language code "smn". It is a Sami language spoken by the Inari Sami of Finland and has about 400 native speakers. It's in the Uralic language family. Wikipedia will be created in T264859. https://en.wikipedia.org/wiki/Inari_Sami | https://iso639-3.sil.org/code/smn | [production]
18:30 <ryankemper> search team's backport deploy is complete [production]
18:30 <ryankemper@deploy1001> Synchronized wmf-config/ProductionServices.php: Config: [[gerrit:632683|cloudelastic: envoy sits in front now (T263073)]] (duration: 00m 58s) [production]
18:29 <ryankemper> Above tests are as expected, syncing changes everywhere: `scap sync-file wmf-config/ProductionServices.php 'Config: [[gerrit:632683|cloudelastic: envoy sits in front now (T263073)]]'` [production]
18:27 <ryankemper> `scap pull`ed onto `mwdebug2001`; talking to cloudelastic via mediawiki from codfw has the expected decrease in latency due to the tls connection pooling [production]
18:24 <ryankemper> `scap pull`ed onto `mwdebug1002`. Talking to cloudelastic on localhost (which routes thru envoy), 6105 is `cloudelastic-chi-eqiad`, 6106 is `cloudelastic-omega-eqiad`, and 6107 is `cloudelastic-psi-eqiad` as expected [production]
18:20 <ryankemper> (backport) HEAD set to 834b4571f978674162fa805906e665e35ac68e27 as expected [production]
18:12 <hashar@deploy1001> Synchronized php-1.36.0-wmf.10/includes/HeaderCallback.php: Preload class used in HeaderCallback - T261260 (duration: 01m 01s) [production]
17:58 <hashar> Pulled https://gerrit.wikimedia.org/r/c/mediawiki/core/+/632680 on deployment staging area and mw2001 [production]
17:35 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
17:33 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:39 <jgleeson> updated civicrm from 39b4f954ed to b86a15a430 [production]
16:35 <mutante> switching webproxy service names to the new local install servers in esams/eqsin/ulsfo T242602 [production]
15:12 <godog> upgrade rsyslog to 8.2008.0-1~bpo10+1 on centrallog1001 - T259780 [production]
14:45 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:41 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
14:37 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:33 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
14:22 <jayme@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
14:04 <hoo> Ran "mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php --wiki=wikidatawiki --property-id P1820 --new-data-type external-id" on mwmaint2001 (T263986) [production]
14:04 <jayme@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
14:03 <akosiaris@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
14:00 <jayme@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'cxserver' for release 'staging' . [production]
13:42 <jayme> updated envoyproxy to 1.15.1-2 on all eqiad hosts [production]
13:39 <akosiaris@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
13:37 <akosiaris@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'test' . [production]
13:37 <akosiaris@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'staging' . [production]
13:18 <volker-e@deploy1001> Finished deploy [design/style-guide@e3fda83]: Deploy design/style-guide: (duration: 00m 04s) [production]
13:18 <volker-e@deploy1001> Started deploy [design/style-guide@e3fda83]: Deploy design/style-guide: [production]