|
2012-08-17
§
|
| 17:00 |
<cmjohnson1> |
shutting down search23 to repair DIMM |
[production] |
| 16:59 |
<cmjohnson1> |
mc1 shutting down to repair DIMM |
[production] |
| 16:31 |
<mark> |
Enabled all 3 new 802.3ad aggregated links for asw-d1-sdtpa to asw-d3-sdtpa |
[production] |
| 16:15 |
<RobH> |
authdns-update to correct typo, nothing really broke badly, i r lucky |
[production] |
| 15:54 |
<RobH> |
fluorine now coming down for reinstall, all log files backed up to nfs |
[production] |
| 15:47 |
<mark> |
Created static 802.3ad trunks on csw1-sdtpa for asw-d1-sdtpa to asw-d3-sdtpa uplinks, deployed with only the existing port enabled |
[production] |
| 15:44 |
<mark> |
Created static 802.3ad trunks on asw-d1-sdtpa and asw-d2-sdtpa for uplinks |
[production] |
| 15:31 |
<RobH> |
puppet is taking forever on nagios, as soon as it completes, will remove fluorine from decom and rerun |
[production] |
| 15:11 |
<RobH> |
authdns-update for fluorine ip update |
[production] |
| 15:04 |
<RobH> |
lvm restructure |
[production] |
| 15:04 |
<RobH> |
all data copied, all traffic moved, fluorine coming down for reinstallation into internal vlan and lvs restructure |
[production] |
| 15:03 |
<RobH> |
all log traffic appears to have moved back to nfs1, copying fluorine logs back to nfs for reformat |
[production] |
| 14:53 |
<mark> |
Setup static 802.3ad trunk on asw-d3-sdtpa with one port up |
[production] |
| 14:27 |
<robh> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 14:27 |
<robh> |
synchronized wmf-config/CommonSettings.php |
[production] |
| 13:48 |
<robh> |
ran sync-common-all |
[production] |
| 13:34 |
<RobH> |
moving apache logs from pushing to fluorine back to nfs1 for now |
[production] |
| 05:12 |
<Tim> |
queued refreshlinks jobs for 1330 commonswiki image description pages with coord templates which weren't refreshed in the previous manual attempt, probably due to failed jobs |
[production] |
| 03:34 |
<aaron> |
synchronized php-1.20wmf9/includes/media/DjVu.php 'deployed 3ae632c3c9111b16f019574bcafec58fa3ee408d' |
[production] |
| 03:34 |
<aaron> |
synchronized php-1.20wmf9/includes/media/SVG.php 'deployed 3ae632c3c9111b16f019574bcafec58fa3ee408d ' |
[production] |
| 03:16 |
<aaron> |
synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php |
[production] |
| 03:14 |
<aaron> |
synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php |
[production] |
| 03:10 |
<aaron> |
synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php 'Profiling' |
[production] |
| 03:08 |
<aaron> |
synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendStore.php 'Profiling' |
[production] |
| 02:42 |
<aaron> |
synchronized php-1.20wmf9/includes/filerepo/backend/SwiftFileBackend.php 'Profiling' |
[production] |
| 02:37 |
<LocalisationUpdate> |
completed (1.20wmf9) at Fri Aug 17 02:37:38 UTC 2012 |
[production] |
| 01:12 |
<aaron> |
synchronized php-1.20wmf9/includes/filerepo/backend/TempFSFile.php 'deployed 08ea8adc95a4edba0b1f2f13c34f2a57725f1a58' |
[production] |
| 01:12 |
<aaron> |
synchronized php-1.20wmf9/includes/filerepo/file/File.php 'deployed 08ea8adc95a4edba0b1f2f13c34f2a57725f1a58' |
[production] |
| 00:13 |
<Tim> |
copying MW log archives from NFS to fluorine via rsync on fenari |
[production] |
|
2012-08-16
§
|
| 23:39 |
<Tim> |
on fluorine: chown'd log files and restarted udp2log to get new uid |
[production] |
| 23:39 |
<binasher> |
reslaved es1002 after conversion to innodb |
[production] |
| 23:31 |
<olivneh> |
synchronized php-1.20wmf9/extensions/E3Experiments/Experiments.hooks.php |
[production] |
| 23:26 |
<mutante> |
fix date/NTP on sq36 |
[production] |
| 23:14 |
<kaldari> |
Finished syncing Wikimedia installation... : |
[production] |
| 23:12 |
<mutante> |
killing nagios-wm. it stopped talking even though stuff gets written to --infile |
[production] |
| 22:51 |
<kaldari> |
Started syncing Wikimedia installation... : |
[production] |
| 22:25 |
<preilly> |
Finished syncing Wikimedia installation... : update mobile frontend |
[production] |
| 22:04 |
<preilly> |
Started syncing Wikimedia installation... : update mobile frontend |
[production] |
| 22:03 |
<notpeter> |
restarting puppet on nfs1/2 |
[production] |
| 21:54 |
<notpeter> |
restarting puppet on locke |
[production] |
| 21:51 |
<notpeter> |
restarting puppet on emery |
[production] |
| 21:39 |
<notpeter> |
restarting puppet on oxygen |
[production] |
| 21:37 |
<cmjohnson1> |
shutting ms-be6 down for hardware testing/replacing |
[production] |
| 21:27 |
<notpeter> |
stopping puppet on nfs2 and restarting both udp2log instances to switch user. will keep puppet off until I merge new init script into puppet |
[production] |
| 21:20 |
<notpeter> |
stopping puppet on nfs1 and restarting both udp2log instances to switch user. will keep puppet off until I merge new init script into puppet |
[production] |
| 21:12 |
<maplebed> |
ms-be6 down for bad disk repair |
[production] |
| 20:58 |
<Ryan_Lane> |
attempting to upgrade virt1000 to precise |
[production] |
| 20:47 |
<cmjohnson1> |
ms-be6 shutting down to replace 2 disks |
[production] |
| 20:38 |
<notpeter> |
stopping puppet on locke and restarting both udp2log instances to switch user. will keep puppet off until I merge new init script into puppet |
[production] |
| 20:12 |
<RobH> |
ms-be1004 is ready to be installed into service |
[production] |