|
2013-01-29
§
|
| 22:02 |
<Ryan_Lane> |
rebooting labstore2 |
[production] |
| 22:01 |
<RobH> |
updating torrus files via puppet, sorry if torrus crashes out, its bitchy |
[production] |
| 22:00 |
<notpeter> |
temp stopping puppet on brewster |
[production] |
| 21:58 |
<Ryan_Lane> |
rebooting labstore1 |
[production] |
| 21:57 |
<Ryan_Lane> |
stopping all gluster services |
[production] |
| 21:38 |
<cmjohnson1> |
authdns update adding mgmt ip's to zone files for hpm misc servers in eqiad |
[production] |
| 21:33 |
<Ryan_Lane> |
stopping gluster services on labstore1, this will affect nfs mounts (like ssh keys and public data exports) |
[production] |
| 20:43 |
<RobH> |
updated rt config, pmtpa comments now come from pmtpa-comment, not rt-comment to match the rest of the queues configuration |
[production] |
| 19:51 |
<cmjohnson1> |
ms-be1004 going down for h/w replacement |
[production] |
| 19:47 |
<hashar> |
jenkins: updating all jobs |
[production] |
| 19:31 |
<cmjohnson1> |
ms-be1003 going down to replace h/w |
[production] |
| 19:29 |
<Ryan_Lane> |
upgrading glusterfs to 3.3.1 from 3.3.0 and replacing old glusterfs package with new packages |
[production] |
| 19:09 |
<cmjohnson1> |
ms-be1002 going down to replace h/w |
[production] |
| 18:37 |
<cmjohnson1> |
ms-be1001 going down to replace h/w |
[production] |
| 17:57 |
<andrewbogott> |
restarted apache and memcached on virt0 (apparently fixing main page errors) |
[production] |
| 17:10 |
<demon> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 14:14 |
<mark> |
powercycled sodium |
[production] |
| 08:20 |
<pgehres> |
synchronized php-1.21wmf8/extensions/CentralNotice/ 'Updating CentralNotice to master' |
[production] |
| 04:58 |
<tstarling> |
synchronized php-1.21wmf6/cache/trusted-xff.cdb |
[production] |
| 04:57 |
<tstarling> |
synchronized php-1.21wmf7/cache/trusted-xff.cdb |
[production] |
| 04:53 |
<tstarling> |
synchronized php-1.21wmf8/cache/trusted-xff.cdb |
[production] |
| 04:51 |
<tstarling> |
synchronized php-1.21wmf8/cache/trusted-xff.cdb |
[production] |
| 04:47 |
<tstarling> |
synchronized php-1.21wmf8/cache/trusted-xff.cdb |
[production] |
| 04:36 |
<tstarling> |
synchronized php-1.21wmf8/cache/trusted-xff.cdb 'fixing corrupted CDB file' |
[production] |
| 04:36 |
<tstarling> |
synchronized php-1.21wmf7/cache/trusted-xff.cdb 'fixing corrupted CDB file' |
[production] |
| 02:55 |
<LocalisationUpdate> |
completed (1.21wmf7) at Tue Jan 29 02:55:44 UTC 2013 |
[production] |
| 02:30 |
<LocalisationUpdate> |
completed (1.21wmf8) at Tue Jan 29 02:30:42 UTC 2013 |
[production] |
| 02:17 |
<Ryan_Lane> |
deleting sysadmin and netadmin roles from ldap |
[production] |
| 01:55 |
<Ryan_Lane> |
adding ldap entries for combined sysadmin/netadmin role, projectadmin |
[production] |
| 01:29 |
<K4-713> |
updated payments cluster to ec4d3f24862559 |
[production] |
| 00:44 |
<RobH> |
typo, knsq18 |
[production] |
| 00:44 |
<RobH> |
squid backend on knsq48 rebuilding |
[production] |
| 00:36 |
<RobH> |
crazy memory and cpu spike on knsq18 for no apparent reason, restarting process (memory leak?) |
[production] |
|
2013-01-28
§
|
| 23:27 |
<K4-713> |
synchronized payments cluster to 1948266b33e905b |
[production] |
| 22:40 |
<pgehres> |
synchronized wmf-config/CommonSettings.php 'Enabling CN:Translate on testwiki' |
[production] |
| 22:37 |
<hashar> |
jenkins: applied pep8 and pyflakes jobs to sartoris repository. |
[production] |
| 22:35 |
<pgehres> |
Finished syncing Wikimedia installation... : Dark deploying Translate integration with CentralNotice |
[production] |
| 22:20 |
<pgehres> |
Started syncing Wikimedia installation... : Dark deploying Translate integration with CentralNotice |
[production] |
| 21:26 |
<pgehres> |
synchronized php-1.21wmf8/extensions/LandingCheck 'Updating LandingCheck to master' |
[production] |
| 20:42 |
<LeslieCarr> |
cleared a lot of 0 size badly named logs on flourine:/a/mw-log |
[production] |
| 20:41 |
<sbernardin1> |
taking down osm-web1 for h/w installation |
[production] |
| 19:25 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 19:09 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: enwiki to 1.21wmf8 |
[production] |
| 19:05 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 18:31 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 18:08 |
<paravoid> |
upgrading ffmpeg/libav for USN-1705-1/USN-1706-1 on all image/video scalers in eqiad/pmtpa |
[production] |
| 18:00 |
<sbernardin1> |
shutting down ms-be11 for h/w replacement |
[production] |
| 15:29 |
<paravoid> |
upgraded & rebooting emery, 208 days bug |
[production] |
| 15:06 |
<mark> |
Pooled cp3009 and cp3010 in the esams upload pool |
[production] |
| 02:52 |
<LocalisationUpdate> |
completed (1.21wmf7) at Mon Jan 28 02:52:53 UTC 2013 |
[production] |