2501-2550 of 10000 results (50ms)
2017-11-07 §
14:01 <moritzm> rebooting mw canaries to 4.9.51 kernel (also picking up openssl/openssl1.1 updates) [production]
13:50 <herron> rebooted fermium (lists) for kernel update [production]
13:46 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Add comment about db1105 current status (duration: 00m 47s) [production]
13:43 <herron> starting rolling mx reboots for kernel update [production]
13:31 <marostegui> Deploy schema change on s5 codfw master (db2023) with replication, this will generate lag on codfw - T174569 [production]
13:19 <moritzm> rebooting naos for kernel update [production]
13:17 <gehel> reboot maps eqiad cluster for upgrades [production]
13:10 <moritzm> rebooting wasat for kernel update [production]
12:50 <mutante> hafnium, tungsten: groupdel perf-roots to go with gerrit:389663 (T179728) [production]
12:00 <mobrovac@tin> Finished deploy [restbase/deploy@eab2948]: Use the new storage for wikidata.org - T179417 (duration: 08m 14s) [production]
11:52 <mobrovac@tin> Started deploy [restbase/deploy@eab2948]: Use the new storage for wikidata.org - T179417 [production]
11:43 <mobrovac> restbase truncating cassandra 2 non-WP tables for T179417 [production]
11:42 <mobrovac> restbase truncating cassandra 2 non-WP tables for T179420 [production]
11:29 <moritzm> installing java security updates/restarting cassandra on restbase2001 (cassandra3 node) [production]
11:07 <ema> cache_misc rolling reboots: upgrading kernel to 4.9.51, libssl to 1.0.2m and 1.1.0g [production]
10:59 <ema> reboot cp3007: upgrading kernel to 4.9.51, libssl to 1.0.2m and 1.1.0g [production]
10:59 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Pool db2092 as recentchanges multi-instance host on s1 and s3 T178359 (duration: 00m 45s) [production]
10:58 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Pool db2092 as recentchanges multi-instance host on s1 and s3 T178359 (duration: 00m 45s) [production]
10:41 <moritzm> restarting ntpd on dns recursors to pick up openssl update [production]
10:28 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Pool db2091 as recentchanges multi-instance host on s2 and s4 T178359 (duration: 00m 45s) [production]
10:27 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Pool db2091 as recentchanges multi-instance host on s2 and s4 T178359 (duration: 00m 46s) [production]
10:24 <elukey> create staging database on db1108 (researchers scratch pad) - T177405 [production]
10:19 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Pool db2086 as recentchanges multi-instance host on s5 and s7 T178359 (duration: 00m 45s) [production]
10:09 <gehel> reboot of maps codfw cluster for upgrades [production]
09:59 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Pool db2086 as recentchanges multi-instance host on s5 and s7 T178359 (duration: 00m 45s) [production]
09:58 <paravoid> updating certspotter to 0.5 in apt and tegmen/einsteinium [production]
09:40 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Pool db2089 as recentchanges multi-instance host on s6 and s5(s8) T178359 (duration: 00m 46s) [production]
09:39 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Pool db2089 as recentchanges multi-instance host on s6 and s5(s8) T178359 (duration: 00m 45s) [production]
09:13 <marostegui> Stop MySQL on db1038 - host to be decommissioned - T177911 [production]
09:11 <moritzm> installing java security updates/restarting cassandra on restbase2002 [production]
09:02 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1038 from config - T177911 (duration: 00m 45s) [production]
09:02 <ppchelko@tin> Finished deploy [cpjobqueue/deploy@3db0cc4]: Do not set Host header for requests to jobrunner (duration: 00m 49s) [production]
09:02 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1038 from config - T177911 (duration: 00m 47s) [production]
09:01 <ppchelko@tin> Started deploy [cpjobqueue/deploy@3db0cc4]: Do not set Host header for requests to jobrunner [production]
08:49 <marostegui> Run redact_sanitarium for hifwiktionary on db1095 (sanitarium) - T173647 [production]
08:48 <marostegui> Optimize pagelinks and templatelinks on s7 master - db1062 - T174509 [production]
07:18 <mobrovac@tin> Started restart [electron-render/deploy@8dd5f13]: Electron stuck, restarting - T174916 [production]
07:11 <ema> reboot pinkunicorn for kernel (4.9.51) and openssl (1.0.2m) upgrades [production]
06:19 <marostegui> Deploy alter table on s7 codfw master (db2029) with replication, this will cause lag in codfw - T174569 [production]
02:29 <l10nupdate@tin> ResourceLoader cache refresh completed at Tue Nov 7 02:29:54 UTC 2017 (duration 6m 39s) [production]
02:23 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.6) (duration: 07m 19s) [production]
2017-11-06 §
22:34 <bawolff> Deploy patch for T178451 [production]
21:48 <arlolra> Updated Parsoid to 6cb77104 (T179579, T175792) [production]
21:42 <arlolra@tin> Finished deploy [parsoid/deploy@d1df3c3]: (no justification provided) (duration: 08m 46s) [production]
21:35 <awight> Begin stress testing ores* (non-production) [production]
21:33 <arlolra@tin> Started deploy [parsoid/deploy@d1df3c3]: (no justification provided) [production]
21:20 <XioNoX> re-activating BGP session to telia in ulsfo [production]
21:16 <ladsgroup@tin> Finished deploy [ores/deploy@97a1d80]: Deploying early November (T179837) (duration: 10m 47s) [production]
21:05 <ladsgroup@tin> Started deploy [ores/deploy@97a1d80]: Deploying early November (T179837) [production]
20:56 <reedy@tin> Synchronized wmf-config/interwiki.php: Update IW Map again (duration: 00m 46s) [production]