|
2014-03-31
§
|
| 16:08 |
<mutante> |
DNS update - remove tampa search pools (a second time, was duplicate, heh) |
[production] |
| 16:00 |
<bd808> |
Restarted logstash service on deployment-logstash1; no new log events seen since 2014-03-28T10:57 |
[releng] |
| 15:58 |
<bd808> |
Updated kibana on deployment-logstash1 to e317bc6 |
[releng] |
| 15:56 |
<hashar_> |
Cluster slow because some CirrusSearch job is spamming simplewiki . Gotta find a way to throttle the number of jobs being run on jobrunner01 or add more apache boxes . It is transient anyway, might look at limiting the runs tonight |
[releng] |
| 15:33 |
<Jeff_Green> |
reenable fundraising services, reenable silicon icinga monitoring |
[production] |
| 15:10 |
<hashar_> |
Rebased puppet repository. Only one hack left: https://gerrit.wikimedia.org/r/#/c/119534/ |
[releng] |
| 14:28 |
<bd808> |
Started logstash on logstash1001 |
[production] |
| 14:23 |
<bd808> |
Elasticsearch upgraded to 1.0.1 on logstash100[123] |
[production] |
| 14:20 |
<hashar> |
deleting deployment-parsoidcache01 cache the hardway: stopping varnish, deleting files in /srv/vdb/ , starting varnish |
[releng] |
| 14:18 |
<hashar> |
beta cluster DNS entries migrated to point to the EQIAD datacenter. Keeping pmtpa instances around for a couple days |
[production] |
| 14:16 |
<bd808> |
Stopped elasticsearch on logstash100[123] |
[production] |
| 14:13 |
<bd808> |
Stopped logstash on logstash1001 |
[production] |
| 14:05 |
<hashar> |
shutdowning database and apache boxes for now. |
[releng] |
| 14:03 |
<hashar> |
shutdowning varnishes instances in pmtpa |
[releng] |
| 13:56 |
<hashar> |
Deleted deployment-cache-upload01 , replaced by deployment-cache-upload02 |
[releng] |
| 13:52 |
<hashar> |
upload varnish cache working :-] |
[releng] |
| 13:48 |
<Jeff_Green> |
icinga alerts disabled for silicon |
[production] |
| 13:47 |
<hashar> |
applying role::cache::upload to role-cache-upload02 |
[releng] |
| 13:37 |
<hashar> |
migrating deployment-cache-upload02.eqiad.Wmflabs to self puppet/salt master |
[releng] |
| 13:34 |
<Jeff_Green> |
fundraising-system full-stop for hardware repairs on queue server |
[production] |
| 13:22 |
<hashar> |
Creating deployment-cache-upload02 to replace deployment-cache-upload01 which was missing the security group "web" |
[releng] |
| 11:57 |
<hashar> |
Jenkins: updating jslint jobs to run a PHP based json linter, will bails out whenever json files are invalid. {{bug|58279}} {{gerrit|113958}} |
[production] |
| 11:30 |
<hashar> |
Update DNS entries to point to EQIAD instances (aka switching beta cluster to eqiad) |
[releng] |
| 10:48 |
<hashar> |
Jenkins fixed up browsertests jobs. Bundler could not compile gems on the eqiad slaves {{gerrit|122346}} |
[production] |
| 09:46 |
<hashar> |
Jenkins: applied hasBrowserTests label on both labs slave. Unblocks the browser tests which were still tied to a deleted instance {{gerrit|122341}} |
[production] |
| 07:23 |
<faidon> |
synchronized php-1.23wmf20/extensions/SpamBlacklist/SpamBlacklist.php 'revert I694860b - SpamBlacklist' |
[production] |
| 07:22 |
<faidon> |
synchronized php-1.23wmf19/extensions/SpamBlacklist/SpamBlacklist.php 'revert I694860b - SpamBlacklist' |
[production] |
| 05:47 |
<faidon> |
synchronized php-1.23wmf19/extensions/SpamBlacklist/SpamBlacklist.php 'local hack to test memcached traffic increase theory' |
[production] |
| 03:08 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Mon Mar 31 03:08:39 UTC 2014 (duration 8m 38s) |
[production] |
| 02:30 |
<LocalisationUpdate> |
completed (1.23wmf20) at 2014-03-31 02:30:26+00:00 |
[production] |
| 02:13 |
<LocalisationUpdate> |
completed (1.23wmf19) at 2014-03-31 02:13:38+00:00 |
[production] |
|
2014-03-29
§
|
| 17:31 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'touch' |
[production] |
| 17:30 |
<reedy> |
synchronized database lists files: Update size related dblists |
[production] |
| 17:28 |
<reedy> |
updated /a/common to {{Gerrit|I7e9595ce0}}: Prepare GeoData config for Elasticsearch switchover |
[production] |
| 16:54 |
<Reedy> |
afl_namespace tinyint -> int on all medium wikis |
[production] |
| 16:06 |
<hoo> |
800+ connections open on db1021, db1037, db1045 (s5) |
[production] |
| 15:49 |
<Reedy> |
afl_namespace tinyint -> int on all small wikis |
[production] |
| 15:10 |
<Reedy> |
afl_namespace tinyint -> int on mediawikiwiki |
[production] |
| 05:59 |
<csteipp> |
deployed patch for bug63251 for wmf19 and wmf20 |
[production] |
| 03:16 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Sat Mar 29 03:16:24 UTC 2014 (duration 16m 23s) |
[production] |
| 02:36 |
<LocalisationUpdate> |
completed (1.23wmf20) at 2014-03-29 02:36:39+00:00 |
[production] |
| 02:16 |
<LocalisationUpdate> |
completed (1.23wmf19) at 2014-03-29 02:16:33+00:00 |
[production] |
| 00:47 |
<paravoid> |
kill updateSpecial MWScript on terbium & pt-kill updateSpecial frwiki query running for 15m |
[production] |