|
2012-05-04
§
|
| 14:02 |
<notpeter> |
putting srv222 back into pybal pool |
[production] |
| 13:50 |
<notpeter> |
removing srv222 from pybal pool for repartitioning |
[production] |
| 13:43 |
<notpeter> |
putting srv221 back into pybal pool |
[production] |
| 13:30 |
<notpeter> |
removing srv221 from pybal pool for repartitioning |
[production] |
| 13:16 |
<mutante> |
going through sq80 to sq86 (upload), full upgrade & reboot |
[production] |
| 12:56 |
<mutante> |
maximum uptime in the sq* group down to 171 days, so we have like a month now for the rest. stopping upgrades for the moment being. |
[production] |
| 12:54 |
<notpeter> |
starting script to move /usr/local/apache to /a partition on all remaing non-imagescaler apaches |
[production] |
| 12:47 |
<mutante> |
(just) new kernels & reboot - sq45,sq49 (upload) |
[production] |
| 12:30 |
<mark> |
Sending ALL non-european upload traffic to eqiad |
[production] |
| 12:23 |
<mutante> |
(just) new kernels & reboot - sq63 to sq66 (209 days up) |
[production] |
| 12:06 |
<mutante> |
dist-upgrade & kernel & reboot - sq42,sq43 - rebooting upload squids one by one |
[production] |
| 11:48 |
<mutante> |
powercycling srv266 one more time, but now creating RT for it, once already showed CPU issue before it was reinstalled recently |
[production] |
| 11:13 |
<apergos> |
restarted lighty on dataset2 ... about ... half an hour ago. stupid case sensitivity |
[production] |
| 10:02 |
<apergos> |
tossed knsq1 through 7 from squid_knams dsh nodegroups file, prolly lots more cleanup where that came from |
[production] |
| 09:34 |
<mutante> |
dist-upgrade/kernel/reboot: sq37, sq41. rebooting upload squid sq41 |
[production] |
| 08:49 |
<mutante> |
dist-upgrade & new kernel & reboot: sq33, sq36 |
[production] |
| 07:47 |
<mutante> |
preemptive rebooting of sq* servers identified as having > 200 days of uptime |
[production] |
| 02:22 |
<LocalisationUpdate> |
completed (1.20wmf2) at Fri May 4 02:22:42 UTC 2012 |
[production] |
| 02:14 |
<LocalisationUpdate> |
completed (1.20wmf1) at Fri May 4 02:13:58 UTC 2012 |
[production] |
| 00:20 |
<raindrift> |
synchronizing Wikimedia installation... : |
[production] |
| 00:18 |
<raindrift> |
synchronizing Wikimedia installation... : Syncing the PageTriage extension, but only enabling on testwiki |
[production] |
| 00:08 |
<awjrichards> |
synchronized wmf-config/CommonSettings.php 'Adding 'fr' to language codes for mobile feedback' |
[production] |
| 00:06 |
<maplebed> |
moved ms1-3 from the production cluster to the test cluster |
[production] |
|
2012-05-03
§
|
| 23:29 |
<LeslieCarr> |
restarting networking on sq55 |
[production] |
| 23:29 |
<tstarling> |
synchronizing Wikimedia installation... : |
[production] |
| 23:27 |
<LeslieCarr> |
restarting networking on sq54 |
[production] |
| 23:24 |
<LeslieCarr> |
restarting networking on sq53 |
[production] |
| 23:21 |
<LeslieCarr> |
restarting networking on sq52 |
[production] |
| 23:16 |
<LeslieCarr> |
restarting networking on sq51 |
[production] |
| 21:30 |
<notpeter> |
removing srv220 from pybal pool for repartitioning |
[production] |
| 21:29 |
<LeslieCarr> |
switching asw-a4-sdtpa from single uplink to lag |
[production] |
| 21:19 |
<notpeter> |
putting srv219 back into pybal pool |
[production] |
| 21:14 |
<asher> |
synchronized wmf-config/db.php 'setting wgDefaultExternalStore to cluster23' |
[production] |
| 21:09 |
<asher> |
synchronized wmf-config/db.php 'reverting cluster23 change' |
[production] |
| 21:05 |
<asher> |
synchronized wmf-config/db.php 'setting wgDefaultExternalStore to cluster23' |
[production] |
| 21:02 |
<binasher> |
about to move ES writes to cluster23 |
[production] |
| 20:47 |
<notpeter> |
removing srv219 from pybal pool for repartitioning |
[production] |
| 20:37 |
<reedy> |
synchronized php-1.20wmf2/extensions/Collection/Collection.templates.php |
[production] |
| 20:37 |
<reedy> |
synchronized php-1.20wmf1/extensions/Collection/Collection.templates.php |
[production] |
| 19:50 |
<binasher> |
restarted profiling collector post parser.php livehack and stats.db removal |
[production] |
| 19:45 |
<notpeter> |
starting script to move /usr/local/apache to /a partition on all non-imagescaler, non-jobrunner apaches |
[production] |
| 19:42 |
<aaron> |
synchronized php-1.20wmf2/includes/parser/Parser.php 'live-hack out template profiling...again.' |
[production] |
| 19:40 |
<aaron> |
synchronized php-1.20wmf1/includes/parser/Parser.php 'live-hack out template profiling...again.' |
[production] |
| 19:31 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Revert $wgDefaultUserOptions[enotifwatchlistpages] = 1' |
[production] |
| 19:20 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Bug 36316 - Set Add pages I edit to my watchlist to true by default for new users' |
[production] |
| 19:15 |
<reedy> |
synchronized wmf-config/CommonSettings.php '$wgDefaultUserOptions[enotifwatchlistpages] = 1' |
[production] |
| 19:09 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Enable show update markers for some more of the larger wikis' |
[production] |
| 19:00 |
<paravoid> |
powercycling all of sq51-sq62, hanged due to 209 days uptime |
[production] |
| 18:46 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Bug 36092 - Activation of flood flag on vec.wikipedia.org' |
[production] |
| 18:43 |
<paravoid> |
powercycling sq59; inaccessible via either SSH or serial due to load |
[production] |