|
2017-11-07
§
|
| 14:02 |
<moritzm> |
actually holding mw canary reboots until SWAT is over |
[production] |
| 14:01 |
<moritzm> |
rebooting mw canaries to 4.9.51 kernel (also picking up openssl/openssl1.1 updates) |
[production] |
| 13:50 |
<herron> |
rebooted fermium (lists) for kernel update |
[production] |
| 13:46 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add comment about db1105 current status (duration: 00m 47s) |
[production] |
| 13:43 |
<herron> |
starting rolling mx reboots for kernel update |
[production] |
| 13:31 |
<marostegui> |
Deploy schema change on s5 codfw master (db2023) with replication, this will generate lag on codfw - T174569 |
[production] |
| 13:19 |
<moritzm> |
rebooting naos for kernel update |
[production] |
| 13:17 |
<gehel> |
reboot maps eqiad cluster for upgrades |
[production] |
| 13:10 |
<moritzm> |
rebooting wasat for kernel update |
[production] |
| 12:50 |
<mutante> |
hafnium, tungsten: groupdel perf-roots to go with gerrit:389663 (T179728) |
[production] |
| 12:00 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@eab2948]: Use the new storage for wikidata.org - T179417 (duration: 08m 14s) |
[production] |
| 11:52 |
<mobrovac@tin> |
Started deploy [restbase/deploy@eab2948]: Use the new storage for wikidata.org - T179417 |
[production] |
| 11:43 |
<mobrovac> |
restbase truncating cassandra 2 non-WP tables for T179417 |
[production] |
| 11:42 |
<mobrovac> |
restbase truncating cassandra 2 non-WP tables for T179420 |
[production] |
| 11:29 |
<moritzm> |
installing java security updates/restarting cassandra on restbase2001 (cassandra3 node) |
[production] |
| 11:07 |
<ema> |
cache_misc rolling reboots: upgrading kernel to 4.9.51, libssl to 1.0.2m and 1.1.0g |
[production] |
| 10:59 |
<ema> |
reboot cp3007: upgrading kernel to 4.9.51, libssl to 1.0.2m and 1.1.0g |
[production] |
| 10:59 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db2092 as recentchanges multi-instance host on s1 and s3 T178359 (duration: 00m 45s) |
[production] |
| 10:58 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Pool db2092 as recentchanges multi-instance host on s1 and s3 T178359 (duration: 00m 45s) |
[production] |
| 10:41 |
<moritzm> |
restarting ntpd on dns recursors to pick up openssl update |
[production] |
| 10:28 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Pool db2091 as recentchanges multi-instance host on s2 and s4 T178359 (duration: 00m 45s) |
[production] |
| 10:27 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db2091 as recentchanges multi-instance host on s2 and s4 T178359 (duration: 00m 46s) |
[production] |
| 10:24 |
<elukey> |
create staging database on db1108 (researchers scratch pad) - T177405 |
[production] |
| 10:19 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Pool db2086 as recentchanges multi-instance host on s5 and s7 T178359 (duration: 00m 45s) |
[production] |
| 10:09 |
<gehel> |
reboot of maps codfw cluster for upgrades |
[production] |
| 09:59 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db2086 as recentchanges multi-instance host on s5 and s7 T178359 (duration: 00m 45s) |
[production] |
| 09:58 |
<paravoid> |
updating certspotter to 0.5 in apt and tegmen/einsteinium |
[production] |
| 09:40 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db2089 as recentchanges multi-instance host on s6 and s5(s8) T178359 (duration: 00m 46s) |
[production] |
| 09:39 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Pool db2089 as recentchanges multi-instance host on s6 and s5(s8) T178359 (duration: 00m 45s) |
[production] |
| 09:13 |
<marostegui> |
Stop MySQL on db1038 - host to be decommissioned - T177911 |
[production] |
| 09:11 |
<moritzm> |
installing java security updates/restarting cassandra on restbase2002 |
[production] |
| 09:02 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Remove db1038 from config - T177911 (duration: 00m 45s) |
[production] |
| 09:02 |
<ppchelko@tin> |
Finished deploy [cpjobqueue/deploy@3db0cc4]: Do not set Host header for requests to jobrunner (duration: 00m 49s) |
[production] |
| 09:02 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Remove db1038 from config - T177911 (duration: 00m 47s) |
[production] |
| 09:01 |
<ppchelko@tin> |
Started deploy [cpjobqueue/deploy@3db0cc4]: Do not set Host header for requests to jobrunner |
[production] |
| 08:49 |
<marostegui> |
Run redact_sanitarium for hifwiktionary on db1095 (sanitarium) - T173647 |
[production] |
| 08:48 |
<marostegui> |
Optimize pagelinks and templatelinks on s7 master - db1062 - T174509 |
[production] |
| 07:18 |
<mobrovac@tin> |
Started restart [electron-render/deploy@8dd5f13]: Electron stuck, restarting - T174916 |
[production] |
| 07:11 |
<ema> |
reboot pinkunicorn for kernel (4.9.51) and openssl (1.0.2m) upgrades |
[production] |
| 06:19 |
<marostegui> |
Deploy alter table on s7 codfw master (db2029) with replication, this will cause lag in codfw - T174569 |
[production] |
| 02:29 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Nov 7 02:29:54 UTC 2017 (duration 6m 39s) |
[production] |
| 02:23 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.6) (duration: 07m 19s) |
[production] |