|
2017-07-31
§
|
| 14:08 |
<gehel> |
shutting down elastic10(17|18|19) for thermal paste - T168816 |
[production] |
| 14:04 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:363206|Remove wgRevisionSliderAlternateSlider]] (duration: 00m 42s) |
[production] |
| 13:57 |
<zfilipin@tin> |
Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:341127|Remove Wikibase vs Interwikisorting checks (T150183)]] (duration: 00m 43s) |
[production] |
| 13:47 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:368701|Remove WMDE log Channel (T168635)]] (duration: 00m 43s) |
[production] |
| 13:30 |
<chasemp> |
disable puppet for cloud-y things |
[production] |
| 12:56 |
<gehel> |
un-banning elastic1020 since it seems to have impact on cluster performances - T168816 |
[production] |
| 12:39 |
<gehel> |
banning elastic10(17|18|19|20) to prepare for thermal paste - T168816 |
[production] |
| 11:12 |
<marostegui> |
Compress s6 on db1102 - T153743 |
[production] |
| 09:28 |
<marostegui> |
Stop replication on labsdb1009 and labsdb1010 for maintenance - T153743 |
[production] |
| 08:55 |
<elukey> |
update nodejs* on aqs100[56789] to 6.11 - T170790 |
[production] |
| 08:45 |
<marostegui> |
Rename table click_tracking and click_tracking_user_properties on db1089 (s1) - T115982 |
[production] |
| 08:35 |
<marostegui> |
Drop table old_growth on s1 - T115982 |
[production] |
| 07:17 |
<marostegui> |
Stop replication on s7 on db1102 for maintenance - T153743 |
[production] |
| 07:12 |
<marostegui> |
Deploy alter table on db1055 - T166204 |
[production] |
| 07:07 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1055 - T166204 (duration: 00m 52s) |
[production] |
| 02:31 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon Jul 31 02:31:39 UTC 2017 (duration 6m 40s) |
[production] |
| 02:24 |
<l10nupdate@tin> |
scap sync-l10n completed (1.30.0-wmf.11) (duration: 07m 41s) |
[production] |
|
2017-07-28
§
|
| 23:32 |
<mutante> |
puppetmaster2001 - git pulled in /var/lib/git/operations/puppet to sync with puppetmaster1001 - accidentally interrupted puppet-merge |
[production] |
| 20:20 |
<foks> |
removing 2FA from User:SPoore (WMF) |
[production] |
| 19:40 |
<mutante> |
releases2001 - OS install worked this time, could not reproduce grub error, signing puppet cert, initial puppet run (T171917) |
[production] |
| 18:48 |
<chasemp> |
enable and force puppet on labtestservices2001,labtestvirt2001,labtestcontrol2001,labservices1002,labcontrol1002,labnet1002,labvirt1014 and labtestneutron2001 to see a newly installed host get the change instead of a noop |
[production] |
| 18:33 |
<chasemp> |
disabling puppet for labs things for trying out refactor rollout |
[production] |
| 17:18 |
<herron> |
cleaned up core files in mw1209:/var/tmp/core to clear disk alert |
[production] |
| 16:21 |
<andrewbogott> |
apt-get install apache2 on labcontrol1001 and labcontrol1002 for security updates |
[production] |
| 16:19 |
<andrewbogott> |
apt-get install apache2 on silver for security updates |
[production] |
| 16:18 |
<andrewbogott> |
apt-get install apache2 on californium for security updates |
[production] |
| 15:23 |
<jynus> |
upgrading and restarting db1102 |
[production] |
| 14:11 |
<paravoid> |
upgrading rhenium to stretch via dist-upgrade |
[production] |
| 13:04 |
<jynus> |
upgrading and restarting db1095 |
[production] |
| 10:31 |
<jynus> |
upgrading and restarting labsdb1009 and labsdb1011 |
[production] |
| 09:41 |
<elukey> |
re-enable irc-echo on einstenium |
[production] |
| 09:07 |
<moritzm> |
installing apache security updates on puppet masters |
[production] |
| 08:41 |
<elukey> |
stop ircecho on einstenium as puppet-error-shower countermeasure |
[production] |
| 07:56 |
<elukey> |
update nodejs to 6.11 on aqs1004 (testing prod node after beta qa) - T170790 |
[production] |
| 07:52 |
<gehel> |
repooling wdqs1001 (data import completed) |
[production] |
| 07:52 |
<elukey> |
forced mii-tool -r eth0 on analytics1034 to get 1G negotiated speed |
[production] |
| 07:52 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=wdqs1001.eqiad.wmnet |
[production] |
| 06:35 |
<moritzm> |
installing apache security updates on trusty systems |
[production] |
| 02:26 |
<mutante> |
scb1002 - systemctl restart pdfrender - was "connect to address 10.64.16.21 and port 5252: Connection refused" in Icinga since a couple hours (T159922) - recovered |
[production] |
| 02:08 |
<ottomata> |
stat1002: disabled puppet, umounted /tmp, /home and /a, poweroff |
[production] |
| 00:51 |
<mutante> |
releases1001 - rsynced reprepro db data from bromine |
[production] |
| 00:27 |
<mutante> |
bromine sudo -E reprepro clearvanished to deleted unused precise-mediawiki causing reprepro errors |
[production] |