|
2017-01-27
§
|
| 21:27 |
<mobrovac@tin> |
Starting deploy [trending-edits/deploy@e0e32bb]: Restart the service to assess the load of replaying the last 6h T156411 |
[production] |
| 20:24 |
<jynus> |
restart and upgrade mariadb on db1048 |
[production] |
| 20:14 |
<mutante> |
db1029, db1042, analytics1015, analytics1026 - puppet node deactivate, remove from icinga, finish decom (T147313, T149793, T146265) |
[production] |
| 20:07 |
<volans> |
updated watchmouse checks for s5 (de wiki) because Main_Page was deleted, used the localized page instead |
[production] |
| 19:52 |
<mutante> |
db1019 - shutdown -h now (T146265) |
[production] |
| 19:51 |
<mutante> |
db1042 - i came to shut it down .. and noticed it had died (or somebody did it) about 3 hours ago .. there it goes (T149793) |
[production] |
| 18:52 |
<twentyafterfour@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group2 wikis to 1.29.0-wmf.9 |
[production] |
| 18:52 |
<twentyafterfour> |
Rolling forward with group2 to 1.29.0-wmf.9 refs T156364 T154683 |
[production] |
| 17:55 |
<papaul> |
OS installation on mc2019-mc2036 |
[production] |
| 16:08 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: db1072 change IP - T156226 (duration: 00m 40s) |
[production] |
| 16:07 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: db1072 change IP - T156226 (duration: 00m 40s) |
[production] |
| 16:01 |
<jynus> |
submitted wmf-mariadb10_10.0.29-2 for T156373 fix |
[production] |
| 15:48 |
<marostegui> |
Stop mysql and shutdown db1072 for maintenance - T156226 |
[production] |
| 15:45 |
<ema> |
cache_text: ban req.url == "/apple-app-site-association" && obj.status == 404 (T155504) |
[production] |
| 14:13 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Add rack positions for s7 in codfw (duration: 00m 40s) |
[production] |
| 14:12 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add rack positions for s7 in eqiad (duration: 00m 40s) |
[production] |
| 13:44 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add rack positions for s6 in eqiad (duration: 00m 40s) |
[production] |
| 13:44 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Add rack positions for s6 in codfw (duration: 00m 40s) |
[production] |
| 13:31 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add rack positions for s5 in eqiad (duration: 00m 40s) |
[production] |
| 13:30 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Add rack positions for s5 in codfw (duration: 00m 40s) |
[production] |
| 13:14 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Add rack positions for s4 in codfw (duration: 00m 41s) |
[production] |
| 13:13 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add rack positions for s4 in eqiad (duration: 00m 43s) |
[production] |
| 13:11 |
<jynus> |
starting db1048 until db1043-bin.001457:753455353, expect it to stop soon |
[production] |
| 13:00 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add rack positions for s3 in eqiad (duration: 00m 40s) |
[production] |
| 13:00 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Add rack positions for s3 in codfw (duration: 00m 40s) |
[production] |
| 12:24 |
<moritzm> |
upgrading mediawiki canaries to new openssl 1.1 package |
[production] |
| 12:13 |
<moritzm> |
upgrading openjdk-7 packages (security updates) on wdqs cluster |
[production] |
| 11:56 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Add rack positions for s2 in codfw (duration: 00m 47s) |
[production] |
| 11:55 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add rack positions for s2 in eqiad (duration: 00m 59s) |
[production] |
| 11:11 |
<moritzm> |
initial installation of openssl bugfix/security updates |
[production] |
| 10:54 |
<moritzm> |
uploaded openssl 1.0.2k for jessie-wikimedia to carbon |
[production] |
| 10:35 |
<paravoid> |
manually running certspotter -all_time as my user on einstenium (will take a few days to complete) |
[production] |
| 10:21 |
<legoktm> |
added addshore to labs-tools-wikibugs2 gerrit group |
[production] |
| 08:13 |
<moritzm> |
uploaded openssl 1.1.0d packages for jessie-wikimedia to carbon |
[production] |
| 02:58 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Fri Jan 27 02:58:52 UTC 2017 (duration 5m 46s) |
[production] |
| 02:53 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.9) (duration: 14m 14s) |
[production] |
| 02:21 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.8) (duration: 07m 57s) |
[production] |
| 01:20 |
<twentyafterfour> |
deploying hotfix for phabricator refs T154479 |
[production] |
| 00:48 |
<mutante> |
carbon - moved the 1.5TB /srv/"mirrors.off", which used to be mirrors but is now on sodium, into / to that /srv/ can be synced without this |
[production] |
| 00:32 |
<dereckson@tin> |
Synchronized wmf-config/throttle.php: Fix throttle rule for Her Girl Friday + Lenny Unconference (T156278) (duration: 00m 53s) |
[production] |
| 00:12 |
<volans> |
re-enabled puppet (with a temporary fix to keep parsoid-vd and parsoid-vd-client stopped) on ruthenium T156177 |
[production] |
|
2017-01-26
§
|
| 23:49 |
<twentyafterfour@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group2 wikis to 1.29.0-wmf.8 |
[production] |
| 23:37 |
<twentyafterfour@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.29.0-wmf.9 |
[production] |
| 23:33 |
<robh> |
archiva.w.o maint done, uses new LE cert. |
[production] |
| 23:14 |
<robh> |
going to try to convert archiva.wikimedia.org from GS to LE cert. will require rehup of nginx |
[production] |
| 22:58 |
<mutante> |
db1019, db1042 - revoke puppet certs, delete salt keys, schedule icinga downtime, stop services (T149793, T146265) |
[production] |
| 22:56 |
<mutante> |
analytics1015, analytics1026 - puppet node clean (again?) - again having problems to remove decom'ed nodes from Icinga (T147313) |
[production] |
| 22:53 |
<mutante> |
analytics1015, analytics1026 - puppet node clean (again?) - again having problems to remove decom'ed nodes from Icinga |
[production] |
| 22:46 |
<mutante> |
mw1181, mw1272, mw1212, mw1174 - service hhvm restart |
[production] |
| 22:06 |
<mobrovac@tin> |
Finished deploy [trending-edits/deploy@e0e32bb]: Bump replay time to 6h for T156411 (duration: 01m 42s) |
[production] |