|
2012-08-15
§
|
| 12:11 |
<mark> |
Restricted NFS mounting of /vol/root on all NetApps |
[production] |
| 11:39 |
<mark> |
Destroyed aggregate 'labs' on nas1-a |
[production] |
| 10:00 |
<Tim> |
deploying bugfix version of wmerrors |
[production] |
| 09:51 |
<Tim> |
moved old MW debug logs out of /home/wikipedia/logs to avoid confusion, live logs are now on fluorine |
[production] |
| 05:22 |
<Tim> |
moving MediaWiki logs to fluorine |
[production] |
| 05:17 |
<tstarling> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 05:15 |
<tstarling> |
synchronized wmf-config/CommonSettings.php |
[production] |
| 04:38 |
<Tim> |
added rate limit exemption for 27.108.200.111 |
[production] |
| 04:38 |
<tstarling> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 03:00 |
<Tim> |
built a udplog package for precise and uploaded it |
[production] |
| 02:35 |
<tstarling> |
synchronized wmf-config/CommonSettings.php 'enabling Scribunto and CodeEditor on test2wiki' |
[production] |
| 02:28 |
<tstarling> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 02:15 |
<tstarling> |
Finished syncing Wikimedia installation... : |
[production] |
| 02:01 |
<LocalisationUpdate> |
failed: git pull of extensions failed |
[production] |
| 01:46 |
<tstarling> |
Started syncing Wikimedia installation... : |
[production] |
| 01:00 |
<tstarling> |
Started syncing Wikimedia installation... : |
[production] |
| 00:47 |
<tstarling> |
Started syncing Wikimedia installation... : |
[production] |
| 00:39 |
<Tim> |
running scap to for extension-list update |
[production] |
|
2012-08-14
§
|
| 23:53 |
<catrope> |
synchronized wmf-config/CommonSettings.php 'Move VE Parsoid from cadmium to wtp1' |
[production] |
| 21:25 |
<kaldari> |
synchronized php-1.20wmf9/extensions/PageTriage/tools/cleanupPageTriage.php 'adding PageTriage maintainence script' |
[production] |
| 21:05 |
<kaldari> |
Finished syncing Wikimedia installation... : |
[production] |
| 20:45 |
<kaldari> |
Started syncing Wikimedia installation... : |
[production] |
| 20:38 |
<mutante> |
killed duplicate nagios-wm |
[production] |
| 20:20 |
<cmjohnson1> |
db44 swapping out disk1 rt 3424 |
[production] |
| 20:08 |
<ottomata> |
installed precise and puppetized stat1001 |
[production] |
| 19:26 |
<cmjohnson1> |
search23 shutting down to replace DIMM A2 rt 3423 |
[production] |
| 19:00 |
<catrope> |
synchronized wmf-config/InitialiseSettings.php 'Deploying https://gerrit.wikimedia.org/r/19399' |
[production] |
| 18:52 |
<RoanKattouw> |
Running /root/fixownership < /root/badownershipfiles in a screen on ms7 |
[production] |
| 18:14 |
<notpeter> |
stopping puppet on brewster for partman experimentation |
[production] |
| 17:34 |
<catrope> |
synchronized php-1.20wmf9/.gitmodules |
[production] |
| 17:33 |
<RoanKattouw> |
Yesterday's find on ms7 is done, found 1.8 million files with bad ownership. Will run a batch chown on them soon |
[production] |
| 17:32 |
<catrope> |
synchronized php-1.20wmf9/extensions/UploadWizard 'Deploy 9d9189458ad8d966d952d295918847f00e304cb5' |
[production] |
| 09:17 |
<nikerabbit> |
synchronized php-1.20wmf9/extensions/Translate/ 'I18ndeploy: Translate bugs' |
[production] |
| 07:46 |
<tstarling> |
Finished syncing Wikimedia installation... : |
[production] |
| 07:07 |
<tstarling> |
Started syncing Wikimedia installation... : |
[production] |
| 06:48 |
<Tim> |
running scap to push out CodeEditor and Scribunto sources, not enabled yet |
[production] |
| 05:45 |
<olivneh> |
synchronized php-1.20wmf9/extensions/E3Experiments/ 'Update E3Experiments to master' |
[production] |
| 03:34 |
<Tim> |
restarting all apaches to get LuaSandbox extension |
[production] |
| 03:26 |
<Tim> |
on mw8: removed broken package php5-memcached |
[production] |
| 02:45 |
<LocalisationUpdate> |
completed (1.20wmf8) at Tue Aug 14 02:45:30 UTC 2012 |
[production] |
| 02:28 |
<LocalisationUpdate> |
completed (1.20wmf9) at Tue Aug 14 02:28:13 UTC 2012 |
[production] |
| 01:06 |
<Tim> |
restarting apaches to get new wmerrors configuration |
[production] |
|
2012-08-13
§
|
| 23:53 |
<Tim> |
uploaded a new version of the php5-wmerrors package for lucid and precise |
[production] |
| 23:18 |
<mutante> |
removed #wikimedia-tech from /usr/local/bin/start-nagios-bot |
[production] |
| 23:01 |
<mutante> |
killing nagios-wm process which isn't supposed to be on #wikimedia-tech anymore |
[production] |
| 21:58 |
<cmjohnson1> |
running authdns-update for labsdb1 & 2 |
[production] |
| 21:19 |
<RoanKattouw> |
...piping result into ms7:/root/badownershipfiles |
[production] |
| 21:18 |
<RoanKattouw> |
ms7# find /export/upload/wik*/*/{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f,archive,math,temp,timeline} ! -user apache |
[production] |
| 21:17 |
<RoanKattouw> |
Searching for files with wrong ownership on ms7 using |
[production] |
| 21:13 |
<RobH> |
reinstalling virt1004 to see if the raid issue re-appears |
[production] |