|
2015-02-11
§
|
| 20:09 |
<bblack> |
eqiad-upload-https -> back to even weighting |
[production] |
| 19:46 |
<bblack> |
all eqiad-upload-https -> cp1064 |
[production] |
| 19:36 |
<subbu> |
temporarily turn off logging to logstash till logstash isssues are resolved. |
[production] |
| 19:34 |
<bblack> |
repooled cp1070 (eqiad bits) in pybal |
[production] |
| 19:18 |
<bd808> |
restarted elasticsearch on logstash1002 after OOM |
[production] |
| 19:15 |
<mutante> |
moved docroots on zirconium to new logical volume for /srv |
[production] |
| 19:09 |
<godog> |
powerdown graphite1002 T88992 |
[production] |
| 18:20 |
<bd808> |
restarted Elasticsearch on logstash1003; preventative, other nodes restarted today |
[production] |
| 17:59 |
<mutante> |
ran puppet on virt1000 - finished just fine, not sure why icinga said fail |
[production] |
| 17:54 |
<mutante> |
running puppet on ruthenium (last was 2 days ago but also not admin disabled..) |
[production] |
| 17:49 |
<godog> |
logging test |
[production] |
| 10:18 |
<godog> |
restart elasticsearch on logstash1003, OOM |
[production] |
| 09:58 |
<hashar> |
restarting Jenkins to upgrade the Credentials plugin |
[production] |
| 06:29 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1057 (duration: 00m 05s) |
[production] |
| 04:54 |
<springle> |
restarting labsdb1002 https://lists.wikimedia.org/pipermail/labs-l/2015-February/003354.html |
[production] |
| 04:35 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed Feb 11 04:34:30 UTC 2015 (duration 34m 29s) |
[production] |
| 03:21 |
<springle> |
restarting labsdb1001 https://lists.wikimedia.org/pipermail/labs-l/2015-February/003354.html |
[production] |
| 02:48 |
<hoo> |
Manually logged a missing global rights log change entry on meta "Ajraddatz changed global group membership for Benoit Rochon from (none) to OTRS-member with the following comment: [[Special:Diff/11227486|request]]". See also T89205 |
[production] |
| 02:28 |
<LocalisationUpdate> |
completed (1.25wmf16) at 2015-02-11 02:26:55+00:00 |
[production] |
| 02:26 |
<l10nupdate> |
Synchronized php-1.25wmf16/cache/l10n: (no message) (duration: 00m 02s) |
[production] |
| 02:13 |
<LocalisationUpdate> |
completed (1.25wmf15) at 2015-02-11 02:12:40+00:00 |
[production] |
| 02:12 |
<l10nupdate> |
Synchronized php-1.25wmf15/cache/l10n: (no message) (duration: 00m 02s) |
[production] |
| 01:56 |
<krenair> |
Synchronized php-1.25wmf16/includes/UserRightsProxy.php: https://gerrit.wikimedia.org/r/#/c/189879/ - same thing for interwiki user rights logs (duration: 00m 07s) |
[production] |
| 01:52 |
<krenair> |
Synchronized php-1.25wmf16/extensions/CentralAuth/includes/CentralAuthGroupMembershipProxy.php: https://gerrit.wikimedia.org/r/#/c/189888/ - fix lack of global group membership change logging (duration: 00m 05s) |
[production] |
| 01:44 |
<springle> |
puppet disabled on lanbdsb1001 labsdb1002. needs restart |
[production] |
| 00:24 |
<krenair> |
Synchronized php-1.25wmf16/extensions/VisualEditor: https://gerrit.wikimedia.org/r/#/c/189867/ (duration: 00m 06s) |
[production] |
| 00:14 |
<krenair> |
Synchronized php-1.25wmf16/extensions/UploadWizard/resources/mw.ApiUploadFormDataHandler.js: https://gerrit.wikimedia.org/r/#/c/189860/ (duration: 00m 05s) |
[production] |
| 00:03 |
<krenair> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/188554/ (duration: 00m 07s) |
[production] |
|
2015-02-10
§
|
| 23:38 |
<andyrussg> |
Synchronized php-1.25wmf16/extensions/CentralNotice/: Update CentralNotice (duration: 00m 06s) |
[production] |
| 22:00 |
<bblack> |
repooled cp1064 eqiad upload frontends in pybal |
[production] |
| 21:38 |
<bblack> |
repooled cp1065 eqiad text frontend in pybal |
[production] |
| 21:16 |
<bblack> |
rebooting cp1064 for experimental kernel (is depooled) |
[production] |
| 21:04 |
<bblack> |
cp1065 frontend disabled in pybal temporarily |
[production] |
| 21:01 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: fixing upload path for labswiki (duration: 00m 06s) |
[production] |
| 20:51 |
<bblack> |
cp1064 frontend disabled in pybal |
[production] |
| 20:28 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group1 to 1.25wmf16 |
[production] |
| 20:20 |
<mutante> |
restarting phd on iridium (phab) for config change |
[production] |
| 17:10 |
<demon> |
Finished scap: No code changes, bringing silver in as deploy target (duration: 17m 31s) |
[production] |
| 16:53 |
<demon> |
Started scap: No code changes, bringing silver in as deploy target |
[production] |
| 16:52 |
<demon> |
Synchronized wmf-config/wikitech.php: Testing silver sync (duration: 00m 05s) |
[production] |
| 16:24 |
<anomie> |
Synchronized wmf-config/CommonSettings.php: SWAT: Revert "Whitelist application/x-gzip on private wikis to fully allow dia files", wasn't a correct fix for the issue (duration: 00m 05s) |
[production] |
| 16:18 |
<godog> |
temporarily disable puppet on carbon |
[production] |
| 16:15 |
<anomie> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable Quiz extension at cawikibooks [[gerrit:187913]] (duration: 00m 07s) |
[production] |
| 16:13 |
<anomie> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Set $wmgUseFloatedToc to false at dewikivoyage [[gerrit:187408]] (duration: 00m 06s) |
[production] |
| 16:10 |
<godog> |
stop replication on elasticsearch cluster and restart ES on elastic1002 |
[production] |
| 16:07 |
<anomie> |
Synchronized wmf-config/CommonSettings.php: SWAT: Whitelist application/x-gzip on private wikis to fully allow dia files [[gerrit:188557]] (duration: 00m 05s) |
[production] |
| 10:57 |
<hoo> |
Synchronized wmf-config/InitialiseSettings-labs.php: (no message) (duration: 00m 06s) |
[production] |
| 10:43 |
<godog> |
reimage graphite1002 |
[production] |
| 07:10 |
<_joe_> |
restarting HHVM on mw1128, in a deadlock in HPHP::RequestInjectionData::onSessionInit () |
[production] |
| 07:09 |
<_joe_> |
restarting HHVM on mw1139, in a deadlock in HPHP::StatCache::refresh () |
[production] |