|
2015-12-12
§
|
| 21:02 |
<ori@tin> |
Synchronized wmf-config/jobqueue-eqiad.php: I53f13a159: Add rdb100[5-6] to job queue configuration (duration: 00m 31s) |
[production] |
| 19:30 |
<hashar> |
Restarted Zuul |
[production] |
| 17:01 |
<krenair@tin> |
Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/258669/ (duration: 00m 29s) |
[production] |
| 10:12 |
<godog> |
bounce nslcd on tools-submit and stop puppet |
[production] |
| 10:00 |
<dcausse> |
elastic in eqiad: disabling TRACE indexing slowlog for urwiki_content |
[production] |
| 09:44 |
<jynus> |
recreated events on db1057 with sql_bin_log = 0 and restarted replication on db2016 |
[production] |
| 09:15 |
<godog> |
move old elasticsearch logs on elastic1026 out to /var/lib/elasticsearch/log (/ is full) |
[production] |
| 09:06 |
<godog> |
move old elasticsearch logs on elastic1016 out to /var/lib/elasticsearch/log (/ is full) |
[production] |
| 09:01 |
<godog> |
move old elasticsearch logs on elastic1012 out to /var/lib/elasticsearch/log (/ is full) |
[production] |
| 03:02 |
<legoktm> |
ran fixDefaultJsonContentPages.php --wiki=thwiktionary for T108663 |
[production] |
| 02:28 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sat Dec 12 02:28:44 UTC 2015 (duration 6m 53s) |
[production] |
| 02:21 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.8) (duration: 08m 31s) |
[production] |
| 00:29 |
<paravoid> |
restarting slapd on seaborgium with manual hack |
[production] |
|
2015-12-11
§
|
| 23:59 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.8/extensions/CentralNotice: Improve impression diet state machine (duration: 00m 32s) |
[production] |
| 22:24 |
<subbu> |
finished deploying parsoid sha ebd62ab5 |
[production] |
| 22:09 |
<subbu> |
restarted parsoid on wtp1002 as a canary |
[production] |
| 22:03 |
<subbu> |
starting parsoid deploy |
[production] |
| 21:18 |
<twentyafterfour@tin> |
Synchronized php-1.27.0-wmf.8/extensions/OpenStackManager/: deploying https://gerrit.wikimedia.org/r/#/c/258516/ (duration: 00m 28s) |
[production] |
| 20:55 |
<jynus> |
stopping and reconfiguring replication on db2016 (only for some minutes) |
[production] |
| 20:48 |
<jynus> |
setting db1057 as the parent mysql of db2016 for s1 replication |
[production] |
| 20:29 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.8/extensions/Flow/maintenance/FlowPopulateRefId.php: Fix fatals due to missing wiki condition (T117786) (duration: 00m 29s) |
[production] |
| 20:06 |
<jynus> |
restarting and reloading mysql for upgrade and reconfiguration at db1057 |
[production] |
| 19:33 |
<demon@mira> |
Synchronized README: testing (duration: 01m 35s) |
[production] |
| 19:26 |
<legoktm> |
started foreachwiki checkLocalUser.php (CentralAuth) on terbium |
[production] |
| 19:14 |
<demon@tin> |
Synchronized wmf-config/InitialiseSettings.php: CentralAuth logging for Lego (duration: 00m 28s) |
[production] |
| 19:13 |
<mutante> |
creating cygnus.codfw.wmnet on ganeti2001 |
[production] |
| 19:06 |
<demon@tin> |
Synchronized wmf-config/wikitech.php: rm some old ldap stuff. yay hiera! (duration: 00m 28s) |
[production] |
| 18:54 |
<demon@tin> |
Synchronized wmf-config/CommonSettings.php: rm some deprecated profiling config (duration: 00m 28s) |
[production] |
| 18:29 |
<demon@tin> |
Synchronized docroot/: cleanup wgconf vhost config (the sequel) (duration: 00m 29s) |
[production] |
| 18:28 |
<demon@tin> |
Synchronized wmf-config/: cleanup wgconf vhost config (duration: 00m 29s) |
[production] |
| 18:13 |
<demon@tin> |
Synchronized wmf-config/: Remove ability to turn off Cirrus completely. It's a scary switch that would Break Everything. There's much better options to use if you need to tune its load or behavior. (duration: 00m 29s) |
[production] |
| 17:54 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1057 for maintenance (duration: 00m 35s) |
[production] |
| 17:44 |
<jynus> |
rolling restart of codfw's S1 mysqls finished |
[production] |
| 16:36 |
<mark> |
Installed bgp-med branch of operations/debs/pybal on pybal-test2001 |
[production] |
| 16:23 |
<mark> |
moved quagga install from pybal-test2001 to pybal-test2002 |
[production] |
| 14:54 |
<jynus> |
restarting and configuring S1:codfw mysqls (db2016,34,42,48,55,62,69,70) |
[production] |
| 14:40 |
<mark> |
root@pybal-test2001:~# apt-get install quagga |
[production] |
| 14:21 |
<andrewbogott> |
re-imaging promethium |
[production] |
| 10:57 |
<_joe_> |
rolling restart of HHVM across the jobrunners to ease memory consumption (this will go on during the day, hhvm restarts will be distantiated by 2 hours each) |
[production] |
| 10:53 |
<_joe_> |
restarting hhvm on mw1122 in order to collect heap dumps |
[production] |
| 10:35 |
<jynus> |
restarting HHVM on mw1122, mw1145 |
[production] |
| 09:50 |
<jynus> |
importing several tables from master to dbstore1002 and labs, lag will will be slightly affected |
[production] |
| 09:26 |
<_joe_> |
ganeti-rebooting serpens, cannot get into console |
[production] |
| 07:10 |
<dcausse> |
deleting some indexing slow logs on elastic1012 and elastic1016 |
[production] |
| 06:46 |
<gwicke> |
deploy restbase c67a41e9d52: add an expensive title to re-render blacklist, per subbu's request |
[production] |
| 03:21 |
<gwicke> |
upgraded codfw nodes to 2.1.12 |
[production] |
| 02:46 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Fri Dec 11 02:46:13 UTC 2015 (duration 6m 46s) |
[production] |
| 02:39 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.8) (duration: 17m 01s) |
[production] |
| 02:34 |
<gwicke> |
restarted nodetool decommission on restbase1007, as I had to restart one of the stream targets |
[production] |
| 02:15 |
<bblack> |
unified cert upgrades complete |
[production] |