|
2017-07-07
§
|
| 12:20 |
<elukey> |
restart mysql on dbstore1002 - high swap used |
[production] |
| 11:40 |
<moritzm> |
rebooting rdb* servers in codfw for kernel update |
[production] |
| 10:30 |
<gehel> |
restarting elastic1043 (corrupted statistics) |
[production] |
| 09:42 |
<gehel> |
unbanning elastic1020 and 1026 from elasticsearch eqiad |
[production] |
| 09:37 |
<gehel> |
restarting elastic1036 (corrupted statistics) |
[production] |
| 09:30 |
<TabbyCat> |
Global rename of Idh0854 → Garam has finished (T167031) |
[production] |
| 09:24 |
<moritzm> |
installing NTP security updates on trusty hosts |
[production] |
| 09:23 |
<akosiaris> |
schedule a month's worth of downtime for ores100X |
[production] |
| 08:56 |
<moritzm> |
restarting HHVM on app server canaries to pick up libgcrypt and expat updates |
[production] |
| 08:54 |
<_joe_> |
reenabling puppet across the fleet |
[production] |
| 08:52 |
<_joe_> |
restarting apache on all puppetmaster, after a successful puppet run |
[production] |
| 08:39 |
<_joe_> |
disabling puppet across the fleet for enabling directory environments in puppet |
[production] |
| 08:32 |
<moritzm> |
installing expat security updates |
[production] |
| 08:27 |
<TabbyCat> |
Starting global rename of Idh0854 → Garam (T167031) |
[production] |
| 08:23 |
<gehel> |
banning elastic1020 and elastic1026 from elasticsearch eqiad cluster |
[production] |
| 07:55 |
<moritzm> |
installing libgcrypt security updates |
[production] |
| 07:42 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1083 - T166204 (duration: 00m 42s) |
[production] |
| 07:39 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2056 - T169510 (duration: 00m 43s) |
[production] |
| 07:37 |
<marostegui@tin> |
scap failed: average error rate on 3/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) |
[production] |
| 06:49 |
<moritzm> |
rebooting bast3002 for kernel update |
[production] |
|
2017-07-06
§
|
| 23:26 |
<demon@tin> |
Synchronized php-1.30.0-wmf.7/extensions/MobileFrontend/includes/MobileFrontend.hooks.php: Only message box styles should be loaded on editor (duration: 00m 43s) |
[production] |
| 23:15 |
<demon@tin> |
Synchronized wmf-config/InitialiseSettings.php: Wikivoyage projects can show more than 3 related articles (duration: 00m 43s) |
[production] |
| 21:52 |
<ppchelko@tin> |
Finished deploy [changeprop/deploy@e1230e6]: Extend automatic blacklisting T169911 (duration: 01m 09s) |
[production] |
| 21:51 |
<ppchelko@tin> |
Started deploy [changeprop/deploy@e1230e6]: Extend automatic blacklisting T169911 |
[production] |
| 20:27 |
<ejegg> |
turned all SmashPig jobs back on |
[production] |
| 19:51 |
<ejegg> |
updated SmashPig from d4458fa552d066f52a9324c09ff8871fcdb89864 to 523d6dddfb1d703213cdca12bc84b62299d8aebd |
[production] |
| 19:46 |
<ejegg> |
rolled back SmashPig to d4458fa552d066f52a9324c09ff8871fcdb89864 |
[production] |
| 19:44 |
<ejegg> |
updated SmashPig from d4458fa552d066f52a9324c09ff8871fcdb89864 to 523d6dddfb1d703213cdca12bc84b62299d8aebd |
[production] |
| 19:40 |
<ejegg> |
disabled smashpig jobs and donation queue consumer |
[production] |
| 19:23 |
<chasemp> |
labstore2003 time bash restore.sh &> /tmp/restore_7_6_2017v1.log for T169774 |
[production] |
| 19:22 |
<demon@tin> |
Finished scap: Forcing l10n rebuild for James_F, plus some wmf-config cleanup (duration: 17m 22s) |
[production] |
| 19:05 |
<demon@tin> |
Started scap: Forcing l10n rebuild for James_F, plus some wmf-config cleanup |
[production] |
| 18:51 |
<niharika29@tin> |
Synchronized php-1.30.0-wmf.7/extensions/CodeMirror: (no justification provided) (duration: 00m 43s) |
[production] |
| 18:38 |
<niharika29@tin> |
Synchronized php-1.30.0-wmf.7/extensions/CirrusSearch/: Fix metastore.php notices https://gerrit.wikimedia.org/r/#/c/363637/ (duration: 00m 53s) |
[production] |
| 18:31 |
<niharika29@tin> |
Synchronized php-1.30.0-wmf.7/extensions/CirrusSearch/: Fix metastore.php notices https://gerrit.wikimedia.org/r/#/c/363637/ (duration: 00m 54s) |
[production] |
| 18:11 |
<niharika29@tin> |
Synchronized wmf-config/InitialiseSettings.php: Add CodeMirror as a beta feature [mediawiki-config] - https://gerrit.wikimedia.org/r/363497 (duration: 00m 43s) |
[production] |
| 18:03 |
<herron> |
moved ununpentium to exim4-daemon-light - T169794 |
[production] |
| 16:50 |
<demon@tin> |
Synchronized README: Testing testing 1 2 3 (duration: 00m 44s) |
[production] |
| 16:45 |
<godog> |
manually create mwdeploy's new home |
[production] |
| 16:12 |
<godog> |
bounce thumbor to apply https://gerrit.wikimedia.org/r/#/c/363626/ |
[production] |
| 16:02 |
<ema@neodymium> |
conftool action : set/pooled=yes; selector: name=cp4014.ulsfo.wmnet,service=varnish-be |
[production] |
| 15:12 |
<herron> |
extend mx[1,2]001 exim log retention to 60 days - T167333 |
[production] |
| 14:50 |
<moritzm> |
rebooting prometheus1003/1004 for kernel update |
[production] |
| 14:39 |
<moritzm> |
rebooting prometheus2004 for kernel update |
[production] |
| 14:39 |
<ema> |
repool cp4006 |
[production] |
| 14:29 |
<ema> |
restart pybal on lvs4001 T169765 |
[production] |
| 14:26 |
<moritzm> |
rebooting prometheus2003 for kernel update |
[production] |
| 14:20 |
<ema> |
restart pybal on lvs4003 T169765 |
[production] |
| 14:16 |
<godog> |
upgrade labmon to grafana 4.4.1 - T169773 |
[production] |
| 14:08 |
<ema> |
restart pybal on lvs4002 T169765 |
[production] |