3101-3150 of 10000 results (29ms)
2020-10-08 §
10:28 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase-backend,name=restbase1028.eqiad.wmnet [production]
10:27 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase-ssl,name=restbase1028.eqiad.wmnet [production]
10:27 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase,name=restbase1028.eqiad.wmnet [production]
10:26 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=eqiad,cluster=restbase,service=restbase-backend,name=restbase1028.eqiad.wmnet [production]
10:26 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=eqiad,cluster=restbase,service=restbase-ssl,name=restbase1028.eqiad.wmnet [production]
10:26 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=eqiad,cluster=restbase,service=restbase,name=restbase1028.eqiad.wmnet [production]
10:26 <hnowlan> pooling restbase1028,restbase1029,restbase1030 [production]
10:22 <mvolz@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'citoid' for release 'production' . [production]
10:14 <mvolz@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'citoid' for release 'staging' . [production]
09:40 <gehel@cumin1001> START - Cookbook sre.wdqs.data-reload [production]
09:10 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:09 <klausman@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:38 <godog> roll-restart swift-object-replicator on ms-be2* - T261633 [production]
08:19 <kormat> running schema change against s8 in eqiad T259831 [production]
08:19 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:19 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:06 <gehel@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:04 <gehel@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:02 <gehel> repooling wdqs2002 [production]
07:55 <marostegui> Rebuild db2125 from snapshots - T260670 [production]
07:45 <marostegui> Stop MySQL on db1077 to build it from s1 snapshot [production]
07:40 <gehel> depooled wdqs2002 to catch up on lag [production]
07:29 <jayme> updated envoyproxy to 1.15.1-2 on all codfw hosts [production]
07:23 <moritzm> installing pyzmq updates from Buster point release [production]
07:00 <dcausse> depooling wdqs2002 (catching-up lag) [production]
06:57 <dcausse> restart blazegraph on wdqs2002 (stuck) T242453 [production]
06:51 <_joe_> enable notifications for wdqs-ssl-codfw [production]
05:33 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
05:27 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission [production]
04:05 <ejegg> updated fundraising python tools from 5515923ef7 to d4e08c52de [production]
00:31 <tgr_> evening deploys done [production]
00:20 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:632796|Enable logging of session cookie changes in group1 (T264793)]] (again, forgot to rebase the previous time) (duration: 00m 59s) [production]
00:15 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:632796|Enable logging of session cookie changes in group1 (T264793)]] (duration: 00m 57s) [production]
00:03 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:632795|Enable logging of session cookie changes in group0 (T264793)]] (duration: 00m 58s) [production]
2020-10-07 §
23:58 <tgr@deploy1001> Synchronized php-1.36.0-wmf.10/includes/session: Backport: [[gerrit:632685|Log when SessionManager is emitting cookies (T264793)]] (duration: 01m 00s) [production]
23:56 <ryankemper@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=0) [production]
23:56 <ryankemper@cumin1001> START - Cookbook sre.elasticsearch.rolling-reboot [production]
23:56 <ryankemper@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-restart (exit_code=0) [production]
23:55 <ryankemper@cumin1001> START - Cookbook sre.elasticsearch.rolling-restart [production]
21:55 <ryankemper@cumin1001> END (FAIL) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=99) [production]
21:48 <ryankemper@cumin1001> START - Cookbook sre.elasticsearch.rolling-reboot [production]
21:14 <ryankemper@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=0) [production]
20:56 <ryankemper@cumin1001> START - Cookbook sre.elasticsearch.rolling-reboot [production]
20:09 <ebernhardson@deploy1001> Finished deploy [wikimedia/discovery/analytics@7fa787e]: airflow: update mjolnir configuration to reduce max training dataset (duration: 03m 23s) [production]
20:05 <ebernhardson@deploy1001> Started deploy [wikimedia/discovery/analytics@7fa787e]: airflow: update mjolnir configuration to reduce max training dataset [production]
19:36 <mutante> blog post: The latest addition to our family of Wikimedia languages is "Inari Sami" with language code "smn". It is a Sami language spoken by the Inari Sami of Finland and has about 400 native speakers. It's in the Uralic language family. Wikipedia will be created in T264859. https://en.wikipedia.org/wiki/Inari_Sami | https://iso639-3.sil.org/code/smn | [production]
18:30 <ryankemper> search team's backport deploy is complete [production]
18:30 <ryankemper@deploy1001> Synchronized wmf-config/ProductionServices.php: Config: [[gerrit:632683|cloudelastic: envoy sits in front now (T263073)]] (duration: 00m 58s) [production]
18:29 <ryankemper> Above tests are as expected, syncing changes everywhere: `scap sync-file wmf-config/ProductionServices.php 'Config: [[gerrit:632683|cloudelastic: envoy sits in front now (T263073)]]'` [production]
18:27 <ryankemper> `scap pull`ed onto `mwdebug2001`; talking to cloudelastic via mediawiki from codfw has the expected decrease in latency due to the tls connection pooling [production]