|
2012-08-14
§
|
| 21:05 |
<kaldari> |
Finished syncing Wikimedia installation... : |
[production] |
| 20:45 |
<kaldari> |
Started syncing Wikimedia installation... : |
[production] |
| 20:38 |
<mutante> |
killed duplicate nagios-wm |
[production] |
| 20:20 |
<cmjohnson1> |
db44 swapping out disk1 rt 3424 |
[production] |
| 20:08 |
<ottomata> |
installed precise and puppetized stat1001 |
[production] |
| 19:26 |
<cmjohnson1> |
search23 shutting down to replace DIMM A2 rt 3423 |
[production] |
| 19:00 |
<catrope> |
synchronized wmf-config/InitialiseSettings.php 'Deploying https://gerrit.wikimedia.org/r/19399' |
[production] |
| 18:52 |
<RoanKattouw> |
Running /root/fixownership < /root/badownershipfiles in a screen on ms7 |
[production] |
| 18:14 |
<notpeter> |
stopping puppet on brewster for partman experimentation |
[production] |
| 17:34 |
<catrope> |
synchronized php-1.20wmf9/.gitmodules |
[production] |
| 17:33 |
<RoanKattouw> |
Yesterday's find on ms7 is done, found 1.8 million files with bad ownership. Will run a batch chown on them soon |
[production] |
| 17:32 |
<catrope> |
synchronized php-1.20wmf9/extensions/UploadWizard 'Deploy 9d9189458ad8d966d952d295918847f00e304cb5' |
[production] |
| 09:17 |
<nikerabbit> |
synchronized php-1.20wmf9/extensions/Translate/ 'I18ndeploy: Translate bugs' |
[production] |
| 07:46 |
<tstarling> |
Finished syncing Wikimedia installation... : |
[production] |
| 07:07 |
<tstarling> |
Started syncing Wikimedia installation... : |
[production] |
| 06:48 |
<Tim> |
running scap to push out CodeEditor and Scribunto sources, not enabled yet |
[production] |
| 05:45 |
<olivneh> |
synchronized php-1.20wmf9/extensions/E3Experiments/ 'Update E3Experiments to master' |
[production] |
| 03:34 |
<Tim> |
restarting all apaches to get LuaSandbox extension |
[production] |
| 03:26 |
<Tim> |
on mw8: removed broken package php5-memcached |
[production] |
| 02:45 |
<LocalisationUpdate> |
completed (1.20wmf8) at Tue Aug 14 02:45:30 UTC 2012 |
[production] |
| 02:28 |
<LocalisationUpdate> |
completed (1.20wmf9) at Tue Aug 14 02:28:13 UTC 2012 |
[production] |
| 01:06 |
<Tim> |
restarting apaches to get new wmerrors configuration |
[production] |
|
2012-08-13
§
|
| 23:53 |
<Tim> |
uploaded a new version of the php5-wmerrors package for lucid and precise |
[production] |
| 23:18 |
<mutante> |
removed #wikimedia-tech from /usr/local/bin/start-nagios-bot |
[production] |
| 23:01 |
<mutante> |
killing nagios-wm process which isn't supposed to be on #wikimedia-tech anymore |
[production] |
| 21:58 |
<cmjohnson1> |
running authdns-update for labsdb1 & 2 |
[production] |
| 21:19 |
<RoanKattouw> |
...piping result into ms7:/root/badownershipfiles |
[production] |
| 21:18 |
<RoanKattouw> |
ms7# find /export/upload/wik*/*/{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f,archive,math,temp,timeline} ! -user apache |
[production] |
| 21:17 |
<RoanKattouw> |
Searching for files with wrong ownership on ms7 using |
[production] |
| 21:13 |
<RobH> |
reinstalling virt1004 to see if the raid issue re-appears |
[production] |
| 20:55 |
<notpeter> |
updating root auth keys file on ms7 by hand. |
[production] |
| 20:36 |
<catrope> |
synchronized php-1.20wmf9/resources/jquery/jquery.tablesorter.js 'Deploy 1fafaef3aa157f49cbbf4b9cffb03544bcf72a08' |
[production] |
| 20:04 |
<preilly> |
updating zero for custom banner colors |
[production] |
| 20:03 |
<preilly> |
synchronized php-1.20wmf9/extensions/ZeroRatedMobileAccess 'update for landing page' |
[production] |
| 20:03 |
<preilly> |
synchronized php-1.20wmf8/extensions/ZeroRatedMobileAccess 'update for landing page' |
[production] |
| 19:42 |
<preilly> |
synchronized php-1.20wmf8/extensions/ZeroRatedMobileAccess 'update for landing page' |
[production] |
| 19:42 |
<preilly> |
update to zero for banner text link |
[production] |
| 19:41 |
<preilly> |
synchronized php-1.20wmf9/extensions/ZeroRatedMobileAccess 'update for landing page' |
[production] |
| 19:36 |
<preilly> |
synchronized php-1.20wmf8/extensions/ZeroRatedMobileAccess 'update for landing page' |
[production] |
| 19:32 |
<RobH> |
powering off analytics1011-1014 to remove the add on nics (disrupting installer) |
[production] |
| 18:25 |
<aaron> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Moved enwiki to 1.20wmf9 |
[production] |
| 17:21 |
<aaron> |
synchronized wmf-config/filebackend.php 'Switched backend reads to swift for testwikis and mw.org' |
[production] |
| 15:18 |
<paravoid> |
reprovisioning srv281 with a new fs layout |
[production] |
| 13:05 |
<mark> |
Doing NDU of IOM3 shelf firmware on nas1001-a |
[production] |
| 12:59 |
<mark> |
Doing NDU of IOM3 shelf firmware on nas1-a |
[production] |
| 12:47 |
<mark> |
Setup NTP on all NetApps |
[production] |
| 12:24 |
<mark> |
Restored SnapMirror relationship between nas1-a and nas1001-a, 70 days lagged, transfer initiated |
[production] |
| 12:04 |
<paravoid> |
apt: include php-memcached 2.0.1-6~wmf+lucid2 @ lucid-wikimedia |
[production] |
| 11:59 |
<paravoid> |
apt: include php5 5.3.10-1ubuntu3.2+wmf1 @ precise-wikimedia |
[production] |
| 11:08 |
<paravoid> |
apt: remove wikidiff2/0.0.1wm1 (obsolete), copysrc php-wikidiff2 {lucid,precise}-wikimedia |
[production] |