4151-4200 of 10000 results (85ms)
2019-06-03 §
08:01 <marostegui> Remove db1064 from tendril and zarcillo T223217 [production]
07:58 <elukey> refresh field list for logstash (via kibana Management -> Index patterns -> etc..) [production]
07:48 <marostegui> Repool db1103 after upgrade T224852 [production]
07:29 <marostegui> Stop MySQL on db1103 (s2 and s4) for upgrade T224852 [production]
07:29 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1103 for upgrade (duration: 00m 47s) [production]
07:10 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1081 into API after upgrade (duration: 00m 48s) [production]
06:51 <elukey> add the server field to the webrequest event format in varnishkafka + roll restart of all the varnishkafkas (via puppet) - T224236 [analytics]
06:50 <elukey> roll restart varnishkafka (via puppet) for a config change - T224236 [production]
06:46 <kartik@deploy1001> scap-helm cxserver finished [production]
06:46 <kartik@deploy1001> scap-helm cxserver cluster eqiad completed [production]
06:45 <kartik@deploy1001> scap-helm cxserver upgrade -f cxserver-eqiad-values.yaml production stable/cxserver [namespace: cxserver, clusters: eqiad] [production]
06:45 <kartik@deploy1001> scap-helm cxserver finished [production]
06:45 <kartik@deploy1001> scap-helm cxserver cluster codfw completed [production]
06:45 <kartik@deploy1001> scap-helm cxserver upgrade -f cxserver-codfw-values.yaml production stable/cxserver [namespace: cxserver, clusters: codfw] [production]
06:44 <kartik@deploy1001> scap-helm cxserver finished [production]
06:44 <kartik@deploy1001> scap-helm cxserver cluster staging completed [production]
06:44 <kartik@deploy1001> scap-helm cxserver upgrade -f cxserver-staging-values.yaml staging stable/cxserver [namespace: cxserver, clusters: staging] [production]
06:39 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1081 into API after upgrade (duration: 00m 49s) [production]
06:26 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1081 after upgrade (duration: 00m 46s) [production]
06:04 <marostegui> Stop MySQL on db1081 for upgrade - T224852 [production]
06:00 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1081 for upgrade (duration: 00m 47s) [production]
05:50 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool es1019 T213422 (duration: 00m 46s) [production]
05:45 <marostegui> Upgrade mariadb on dbstore1004 - T224852 [production]
05:17 <marostegui> Upgrade MariaDB on codfw hosts in preparation for s4 master failover T217396 [production]
05:15 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to es1019 T213422 (duration: 00m 46s) [production]
05:05 <marostegui> Remove db2037 from tendril and zarcillo T224720 [production]
05:04 <marostegui> Stop MySQL on db2037 for decommission T224720 [production]
04:56 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool es1019 T213422 (duration: 00m 51s) [production]
2019-06-02 §
20:28 <onimisionipe> pooled wdqs1007. It caught up on lag [production]
15:24 <onimisionipe> depooled wdqs1007 to catch up on lags [production]
15:22 <onimisionipe> depool wdqs internal cluster to allow them catch up on lags. depool one at a time [production]
12:36 <wm-bot> <peterbowman> Commit 562513a: Update wiki submodule (JDK 9+ try-w-rsrc) [tools.pbbot]
07:04 <elukey> manually restart refinery-import-page-history-dumps.service with some debug info to check what file breaks [analytics]
04:50 <joal> Restart mediawiki-history-wikitext (dumps conversion) oozie job [analytics]
04:12 <joal> Restart load-cassandra oozie bundle to use analytics user [analytics]
03:09 <andrewbogott> restarting pdns-recursor on cloudservices 1003 and 1004 (but not at the same time) [production]
2019-06-01 §
22:54 <James_F> Reloading Zuul to re-add Kartographer dependency for JsonConfig T224785 [releng]
22:49 <krinkle@deploy1001> Synchronized php-1.34.0-wmf.7/extensions/3D/modules/mmv.3d.js: T224812 / bd4fbfddbe1a0 (duration: 01m 07s) [production]
21:14 <James_F> Reloading Zuul to add phan for CentralNotice and add DannyS712 to whitelist [releng]
20:17 <wm-bot> <lucaswerkmeister> deployed 32b8528d58 (export options, including PagePile export) [tools.quickcategories]
12:04 <wm-bot> <lucaswerkmeister> deployed 7ba32d514c (PagePile support) [tools.quickcategories]
08:03 <elukey> manually restart refinery-sqoop-whole-mediawiki.service after failure [analytics]
2019-05-31 §
23:34 <James_F> Reloading Zuul to remove Kartographer dependency for JsonConfig T224785 [releng]
23:07 <bd808> Increased quota for floating ips (T224272) [sso]
22:59 <bd808> Increased quota for cores, ram, and instances (T224057) [automation-framework]
22:12 <James_F> Reloading Zuul to add phan for LdapAuthentication [releng]
22:06 <wm-bot> <lucaswerkmeister> deployed ebfed74416 (minor error handling improvements) [tools.quickcategories]
21:47 <aaron@deploy1001> Synchronized wmf-config/db-eqiad.php: Set "secret" field in $wgLBFactoryConf for ChronologyProtector HMACs (duration: 00m 47s) [production]
21:46 <aaron@deploy1001> Synchronized wmf-config/db-codfw.php: Set "secret" field in $wgLBFactoryConf for ChronologyProtector HMACs (duration: 00m 50s) [production]
21:18 <James_F> Reloading Zuul to add dependencies for TimedMediaHandler, ReadingLists, JsonConfig, and FlaggedRevs. [releng]