|
2013-07-23
§
|
| 15:57 |
<Jeff_Green> |
stop icinga notifications for db1025 mysql |
[production] |
| 15:57 |
<Jeff_Green> |
stop icinga notifications for db1008 |
[production] |
| 15:21 |
<Jeff_Green> |
dist-upgrade &reboot db1025 |
[production] |
| 15:14 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 15:13 |
<reedy> |
synchronized database lists files: |
[production] |
| 15:13 |
<Jeff_Green> |
dist-upgrade & reboot payments2-4 |
[production] |
| 14:57 |
<Jeff_Green> |
dist-upgrade & reboot payments2 |
[production] |
| 14:47 |
<ottomata> |
stopping puppet on gadolinium, looking into filters causing socat process packet loss |
[production] |
| 14:22 |
<hashar> |
Jenkins: lanthanum has been added as a Jenkins slave node though NO job should be running there.e |
[production] |
| 10:46 |
<mark> |
synchronized wmf-config/squid.php 'Add ssl1005/ssl1006' |
[production] |
| 05:44 |
<paravoid> |
pooling ssl1005/1006 for https/ipv6 |
[production] |
| 05:34 |
<paravoid> |
authdns-update for cr1-ulsfo <-> cr2-eqiad, dns-auth-lb -> nsN(-new), ssl100x fixed IPv6 |
[production] |
| 02:21 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue Jul 23 02:21:20 UTC 2013 |
[production] |
| 02:12 |
<LocalisationUpdate> |
completed (1.22wmf10) at Tue Jul 23 02:12:36 UTC 2013 |
[production] |
| 02:08 |
<LocalisationUpdate> |
completed (1.22wmf11) at Tue Jul 23 02:08:03 UTC 2013 |
[production] |
| 01:01 |
<Ryan_Lane> |
enabled hyperthreading on ssl1005 |
[production] |
| 00:39 |
<Ryan_Lane> |
depooling ssl1005/6 due to ipv6 issues |
[production] |
| 00:39 |
<Ryan_Lane> |
pooled new https node ssl1006.wikimedia.org (with hyperthreading enabled) |
[production] |
| 00:38 |
<Ryan_Lane> |
pooled new https node ssl1005.wikimedia.org |
[production] |
|
2013-07-22
§
|
| 22:31 |
<reedy> |
synchronized php-1.22wmf11/extensions/UploadWizard |
[production] |
| 22:03 |
<cmjohnson1> |
restarting ns2 |
[production] |
| 21:59 |
<cmjohnson1> |
dns update |
[production] |
| 21:47 |
<LeslieCarr> |
restarting powerdns on ns1 |
[production] |
| 21:04 |
<asher> |
synchronized wmf-config/StartProfiler.php 'only randomly profile web reqs in eqiad' |
[production] |
| 20:39 |
<sbernardin> |
hot swapping disk 13 on db78 storage array |
[production] |
| 20:19 |
<paravoid> |
deploying bmc-config fix across all pmtpa/eqiad mgmt |
[production] |
| 19:42 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Everyhting else no wikipedia to 1.22wmf11 |
[production] |
| 19:36 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikiquote, wikinews and wikibooks to 1.22wmf11 |
[production] |
| 19:29 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikiversity, wikivoyage and wiktionary to 1.22wmf11 |
[production] |
| 19:26 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: special, wikimedia, private, fishbowl and closed to 1.22wmf11 |
[production] |
| 19:25 |
<paravoid> |
restarting & repooling cp1047/cp1059 |
[production] |
| 19:22 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Move 3 wikis with UW to 1.22wmf11 |
[production] |
| 16:02 |
<hashar> |
Jenkins back up |
[production] |
| 15:52 |
<hashar> |
jenkins accidentally killed Jenkins :( |
[production] |
| 15:17 |
<hashar> |
gallium / jenkins : blacklisting a bunch of user agent {{gerrit|75105}} |
[production] |
| 14:41 |
<apergos> |
(btw docs would be nice, is that really the right way to kick it?) |
[production] |
| 14:41 |
<apergos> |
shot and restarted gitblit: on antinomy, cd /var/lib/gitblit, java -jar gitblit.jar & (see bug 51769) |
[production] |
| 12:46 |
<paravoid> |
depooling cp1047/cp1059 for further investigation |
[production] |
| 12:28 |
<paravoid> |
restarting cp1046/cp1060's varnish-frontend |
[production] |
| 11:21 |
<hashar> |
Jenkins: deleting -merge jobs ({{bug|51395}} sequentially to avoid deadlocking jenkins {{bug|51802}} |
[production] |
| 11:19 |
<hashar> |
jenkins restarted |
[production] |
| 11:01 |
<hashar> |
restarting Jenkins, some threads are deadlocked ( see {{bug|51802}} ) |
[production] |
| 09:25 |
<paravoid> |
adding new ns0/ns1 service ip static routes to dobson/linne on cr1-sdtpa/cr2-pmtpa |
[production] |
| 07:18 |
<paravoid> |
restarting pybal and manually ipvsadm removing dns_auth services from lvs1/lvs5/lvs1002/lvs1005 |
[production] |
| 02:18 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Mon Jul 22 02:17:55 UTC 2013 |
[production] |
| 02:10 |
<LocalisationUpdate> |
completed (1.22wmf11) at Mon Jul 22 02:10:16 UTC 2013 |
[production] |
| 02:06 |
<LocalisationUpdate> |
completed (1.22wmf10) at Mon Jul 22 02:06:01 UTC 2013 |
[production] |