|
2016-12-20
§
|
| 17:50 |
<volans> |
rcs1002.eqiad.wmnet: python-gevent downgraded from 1.1b6-1~trusty1 to 1.0-1ubuntu1 |
[production] |
| 17:41 |
<ema> |
rcs1001.eqiad.wmnet: python-gevent downgraded from 1.1b6-1~trusty1 to 1.0-1ubuntu1 |
[production] |
| 17:24 |
<reedy@tin> |
Synchronized wmf-config/throttle.php: T153741 (duration: 00m 40s) |
[production] |
| 16:31 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=yes; selector: cluster=restbase,dc=eqiad,name=restbase101[6-8].* |
[production] |
| 16:30 |
<bblack> |
upgrading tons of outdated OS packages on rcs100x |
[production] |
| 16:29 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=yes; selector: cluster=restbase,dc=codfw,name=restbase201.* |
[production] |
| 16:26 |
<marostegui> |
Running optimize table on db1045 for the revision tables as we urgently need some space back on that host - T153739 |
[production] |
| 16:19 |
<mutante> |
notebook1001/1002 - upgraded php packages |
[production] |
| 16:17 |
<jynus> |
drop repl_records_partition events from db1069 |
[production] |
| 16:09 |
<mobrovac> |
change-prop restarting everywhere |
[production] |
| 15:59 |
<mobrovac> |
restarting change-prop on scb1001 |
[production] |
| 15:48 |
<mobrovac@tin> |
Finished deploy [parsoid/deploy@e8f0865]: Deploy of mwApiServer across the wtp fleet (duration: 05m 59s) |
[production] |
| 15:42 |
<mobrovac@tin> |
Starting deploy [parsoid/deploy@e8f0865]: Deploy of mwApiServer across the wtp fleet |
[production] |
| 15:32 |
<mobrovac@tin> |
Finished deploy [parsoid/deploy@e8f0865]: Canary deploy of mwApiServer to wtp[12]001, take two (duration: 00m 37s) |
[production] |
| 15:31 |
<mobrovac@tin> |
Starting deploy [parsoid/deploy@e8f0865]: Canary deploy of mwApiServer to wtp[12]001, take two |
[production] |
| 14:33 |
<mobrovac@tin> |
Finished deploy [parsoid/deploy@1d75b14]: Canary deploy of mwApiServer to wtp[12]001 (duration: 00m 32s) |
[production] |
| 14:33 |
<mobrovac@tin> |
Starting deploy [parsoid/deploy@1d75b14]: Canary deploy of mwApiServer to wtp[12]001 |
[production] |
| 12:04 |
<jynus> |
running redact_sanitarium.sh on db1069 |
[production] |
| 08:27 |
<elukey> |
renamed some log files ($something.1.gz to $something.1a.gz) on cp1008 and rutherium to unblock logrotation and reduce cronspam - T132324 |
[production] |
| 02:29 |
<mutante> |
fermium (lists) - upgrade apache2 | uranium (ganglia-web) - upgrade apache2, openssh, openssl |
[production] |
| 02:26 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Dec 20 02:26:08 UTC 2016 (duration 4m 48s) |
[production] |
| 02:23 |
<mutante> |
osmium - upgrade imagemagick, php5 packages | hafnium - upgrade php5 |
[production] |
| 02:21 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.6) (duration: 06m 41s) |
[production] |
| 02:17 |
<mutante> |
contint1002/2001, einsteinium/tegmen - upgrade php5 | bromine - upgrade php5, libs |
[production] |
| 02:11 |
<mutante> |
auth (yubiauth), phab2001 - upgrade php5 | bohrium (piwik) - upgrade ssh,ssl,php5 |
[production] |
| 02:02 |
<mutante> |
copper - upgraded php5* | krypton - installed all pending upgrades |
[production] |
| 01:45 |
<mutante> |
terbium - upgrade imagemagick | wasat - upgrade php5, imagemagick |
[production] |
| 01:41 |
<mutante> |
tin - upgrading php, imagemagick,.. |
[production] |
| 01:35 |
<mutante> |
mira - upgrading php5 packages |
[production] |
|
2016-12-19
§
|
| 21:41 |
<ejegg|standup> |
updated fundraising dash from 040557d08b476283df7c2e243c71ce4043ffe7c8 to ea4305bc9b2c919d3f828ba542eb32b91b207af7 |
[production] |
| 21:00 |
<nuria@tin> |
Finished deploy [analytics/refinery@ead5b8b]: (no message) (duration: 01m 59s) |
[production] |
| 20:58 |
<nuria@tin> |
Starting deploy [analytics/refinery@ead5b8b]: (no message) |
[production] |
| 19:44 |
<otto@tin> |
Finished deploy [analytics/refinery@711a572]: (no message) (duration: 00m 06s) |
[production] |
| 19:44 |
<otto@tin> |
Starting deploy [analytics/refinery@711a572]: (no message) |
[production] |
| 19:41 |
<nuria@tin> |
Finished deploy [analytics/refinery@711a572]: (no message) (duration: 00m 04s) |
[production] |
| 19:41 |
<nuria@tin> |
Starting deploy [analytics/refinery@711a572]: (no message) |
[production] |
| 19:36 |
<nuria@tin> |
Finished deploy [analytics/refinery@711a572]: (no message) (duration: 00m 04s) |
[production] |
| 19:36 |
<nuria@tin> |
Starting deploy [analytics/refinery@711a572]: (no message) |
[production] |
| 19:35 |
<nuria@tin> |
Finished deploy [analytics/refinery@711a572]: (no message) (duration: 27m 10s) |
[production] |
| 19:16 |
<mutante> |
analytics1027 - out of disk, apt-get clean to free about 500M |
[production] |
| 19:08 |
<nuria@tin> |
Starting deploy [analytics/refinery@711a572]: (no message) |
[production] |
| 16:17 |
<marostegui> |
Run lots os small optimize tables on db1015 as it needs to get some space back urgently |
[production] |
| 16:04 |
<andrewbogott> |
upgrading to python-urllib3_1.19 on scb1001 |
[production] |
| 14:02 |
<jynus> |
deploying new firewall rules to labsdb1009/10/11 |
[production] |
| 13:39 |
<elukey> |
Manually raise hhvm.server.connection_timeout_seconds on mw1259 to one day |
[production] |
| 13:15 |
<_joe_> |
restarted hhvm, apache on mw1260, raised the apache timeout to 1 day, restarted the jobrunner, disabled puppet |
[production] |
| 11:47 |
<_joe_> |
disabling puppet, reconfiguring timeout on apache, restarting HHVM on mw1259 |
[production] |
| 10:16 |
<elukey> |
reimaging mw1168 and mw1169 to Trusty - T153488 |
[production] |
| 09:38 |
<elukey> |
stopping jobrunner/jobchron daemons on mw116[89] as prep step for repurpose to videoscalers - T153488 |
[production] |
| 09:23 |
<marostegui> |
Stop mysql db2048 (depooled) for maintenance - T149553 |
[production] |