|
2013-09-11
§
|
| 18:25 |
<LeslieCarr> |
moving all external traffic to come in via eqiad |
[production] |
| 18:10 |
<LeslieCarr> |
doing some network maintenance on cr2-pmtpa |
[production] |
| 17:57 |
<RobH> |
forgot to log testsearch1003 is back |
[production] |
| 17:49 |
<RobH> |
testsearch1002 back up, 1003 down |
[production] |
| 17:39 |
<RobH> |
testsearch1001 back in pool, 1002 rebooting for fix |
[production] |
| 17:21 |
<RobH> |
rebooting testsearch1001 to troubleshoot |
[production] |
| 17:10 |
<aaron> |
synchronized wmf-config/filebackend.php |
[production] |
| 17:02 |
<aaron> |
synchronized wmf-config/filebackend.php 'Removed ceph backend config for decommissioning' |
[production] |
| 16:39 |
<demon> |
synchronized wmf-config/InitialiseSettings.php 'Cirrus as default on mw.org, Lucene as secondary' |
[production] |
| 10:18 |
<apergos> |
set php memory limit to 256 on nickel, ori reducd the number of metrics for statsd (see https://gerrit.wikimedia.org/r/#/c/83777/), restarted all of apache, gmond, gmetad and still see the occasional php Allowed memory size exhausted error |
[production] |
| 02:08 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed Sep 11 02:08:28 UTC 2013 |
[production] |
| 02:02 |
<LocalisationUpdate> |
completed (1.22wmf15) at Wed Sep 11 02:02:32 UTC 2013 |
[production] |
| 02:01 |
<LocalisationUpdate> |
completed (1.22wmf16) at Wed Sep 11 02:01:46 UTC 2013 |
[production] |
|
2013-09-10
§
|
| 23:46 |
<LeslieCarr> |
rebooted neon, had crashed |
[production] |
| 23:33 |
<dzahn> |
synchronized ./wmf-config/throttle.php |
[production] |
| 23:32 |
<dzahn> |
synchronized ./wmf-config/throttle.php |
[production] |
| 23:28 |
<springle> |
starting mariadb 10 multi-source replication test from all clusters on db1045 |
[production] |
| 22:24 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 22:15 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Wikivoyage and wikidatawiki to 1.22wmf16 |
[production] |
| 22:12 |
<Jeff_Green> |
dist-upgrade and reboot boron |
[production] |
| 22:09 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue Sep 10 22:08:58 UTC 2013 |
[production] |
| 22:07 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 22:04 |
<reedy> |
synchronized php-1.22wmf16/extensions/Wikibase |
[production] |
| 21:50 |
<LocalisationUpdate> |
completed (1.22wmf15) at Tue Sep 10 21:50:32 UTC 2013 |
[production] |
| 21:50 |
<reedy> |
synchronized php-1.22wmf16/includes/search/SearchEngine.php |
[production] |
| 21:33 |
<mutante> |
temp. hacked a fork limit into apache-graceful-all so my local ssh agent doesn't die when it's too fast, bleh:p |
[production] |
| 21:26 |
<LocalisationUpdate> |
completed (1.22wmf16) at Tue Sep 10 21:26:17 UTC 2013 |
[production] |
| 21:03 |
<csteipp> |
synchronized php-1.22wmf15/extensions/CentralAuth |
[production] |
| 20:56 |
<demon> |
synchronized php-1.22wmf16/extensions/CirrusSearch 'Logging for annoying parser output missing bug' |
[production] |
| 20:55 |
<LeslieCarr> |
setting AMSIX Local pref to 110 due to overloading. high risk |
[production] |
| 20:42 |
<csteipp> |
synchronized php-1.22wmf15/extensions/CentralAuth |
[production] |
| 20:42 |
<csteipp> |
synchronized php-1.22wmf16/extensions/CentralAuth |
[production] |
| 18:43 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Disable ContributionReporting' |
[production] |
| 18:11 |
<reedy> |
synchronized php-1.22wmf15/extensions/Wikibase |
[production] |
| 18:10 |
<reedy> |
synchronized php-1.22wmf15/extensions/WikibaseDataModel |
[production] |
| 18:09 |
<reedy> |
synchronized php-1.22wmf14/extensions/Wikibase |
[production] |
| 18:09 |
<reedy> |
synchronized php-1.22wmf14/extensions/WikibaseDataModel |
[production] |
| 17:52 |
<cmjohnson1> |
removing sq42 from pybal |
[production] |
| 17:20 |
<cmjohnson1> |
rebooting sq42 |
[production] |
| 17:05 |
<apergos> |
ganglia was failign to serve some urls, complaining that memory size of 134217728 bytes exhausted, I upped to 192mb temporarily in /etc/php5/apache2/php.ini, needs followup |
[production] |
| 16:45 |
<reedy> |
synchronized wmf-config/interwiki.cdb 'Updating interwiki cache' |
[production] |
| 16:12 |
<apergos> |
rebooting labstore3 after restart of nfsd did not do the trick |
[production] |
| 12:23 |
<Nemo_bis> |
labstore3 overloaded, beta & tools &co. down |
[production] |
| 12:21 |
<apergos> |
rebooted neon, it was out to lunch |
[production] |
| 01:07 |
<awight> |
crm updated from 75d31d3443e4 to aa79ff40f3ce153 |
[production] |
| 00:40 |
<mutante> |
restarting gmetad on nickel |
[production] |
| 00:27 |
<LeslieCarr> |
restarted gmetad on nickel |
[production] |
| 00:21 |
<csteipp> |
synchronized php-1.22wmf15/extensions/CentralAuth 'Syncing correct submodule checkout of centralauth' |
[production] |