2101-2150 of 10000 results (17ms)
2013-02-18 §
05:10 <Tim> on db1043: running alter table to convert the relevant field to signed, since I figure this is less disruptive than sql_mode = 'NO_UNSIGNED_SUBTRACTION' and less likely to break again than skipping the query [production]
04:58 <Tim> replication broken on db1043, possibly others, due to update of article_feedback_revisions with negative value for afr_total [production]
03:40 <tstarling> synchronized wmf-config/CommonSettings.php [production]
02:29 <LocalisationUpdate> completed (1.21wmf9) at Mon Feb 18 02:29:47 UTC 2013 [production]
2013-02-17 §
02:29 <LocalisationUpdate> completed (1.21wmf9) at Sun Feb 17 02:29:31 UTC 2013 [production]
2013-02-16 §
13:38 <apergos> restarted lucene-search on search1015 about 20 minutes ago [production]
02:28 <LocalisationUpdate> completed (1.21wmf9) at Sat Feb 16 02:28:26 UTC 2013 [production]
01:47 <reedy> synchronized php-1.21wmf9/ [production]
01:12 <reedy> synchronized wmf-config/filebackend.php [production]
2013-02-15 §
23:56 <RobH> authdns-update to correct mw110 reverse entry [production]
23:28 <Ryan_Lane> *modified them in LDAP [production]
23:28 <Ryan_Lane> modified project objects to indicate whether or not they are using gluster volumes for project and/or home storage [production]
23:04 <RobH> disregard mw1161 flaps, im working on it [production]
20:15 <Ryan_Lane> restarting opendj and pdns on virt0 and virt1000 [production]
20:15 <Ryan_Lane> removed opendj from nfs1/2 [production]
19:07 <RobH> no one sign mw86-mw125 puppet certs, im still working on them [production]
18:23 <RobH> mw87-mw125 installing [production]
17:33 <binasher> reimaging db1029-1031 [production]
16:55 <mutante> removing mw1085 from dsh groups - broken hardware (RT-4453) [production]
14:31 <mutante> wikimediafoundation.info activated to redirect .org [production]
14:28 <mutante> gracefulling Apaches in eqiad using dsh [production]
14:25 <dzahn> gracefulled all apaches [production]
14:15 <^demon> [gerrit] updated replication plugin to 20c6fc7e (1.1-SNAPSHOT) which is back in line with upstream [production]
08:02 <olivneh> synchronized php-1.21wmf9/extensions/Echo 'Syncing Iab077a77aa75a6ff0cf03acecca38403595dda93 to fix exception 640c60c4' [production]
02:30 <LocalisationUpdate> completed (1.21wmf9) at Fri Feb 15 02:30:05 UTC 2013 [production]
01:19 <paravoid> deploying squid changes for s/knams/esams/ for knsq* squids (see git log for rationale) [production]
01:09 <olivneh> synchronized wmf-config/CommonSettings.php 'Enabling Schema NS on test2wiki' [production]
2013-02-14 §
22:49 <RobH> mw1194 online and now in service as api apache [production]
22:48 <RobH> mw1200 added to api pool, online and in service [production]
22:44 <RobH> mw1189-1199 (sans 1194) now online as eqiad api apaches [production]
22:37 <mflaschen> Finished syncing Wikimedia installation... : [production]
22:29 <mflaschen> Started syncing Wikimedia installation... : [production]
22:21 <sbernardin> taking storage1 offline for decommissioning per: https://rt.wikimedia.org/Ticket/Display.html?id=4529 [production]
22:18 <RobH> pushing mw1161-1188 into apache service (sans 1161/1164) from false to true in pybal, otherwise all online [production]
22:16 <olivneh> synchronized wmf-config/CommonSettings.php 'Updating for test2wiki' [production]
22:14 <olivneh> synchronized php-1.21wmf9/extensions/Math [production]
22:03 <Ryan_Lane> replacing gluster upstarts on all labstore nodes with a regular init script [production]
22:01 <reedy> synchronized docroot [production]
22:00 <olivneh> synchronized wmf-config [production]
21:49 <olivneh> synchronized wmf-config/InitialiseSettings.php 'Enabling PostEdit on {be|mediawiki|outreach}wikis' [production]
21:45 <olivneh> synchronized php-1.21wmf9/extensions/EventLogging [production]
21:39 <sbernardin> taking down srv266 and srv268 for decommissioning ... https://rt.wikimedia.org/Ticket/Display.html?id=4534 [production]
21:31 <ottomata1> restarted solr jetty on solr2 [production]
21:30 <bsitu> Finished syncing Wikimedia installation... : Update ArticleFeedbackv5, Echo, PageTriage to master [production]
21:25 <ottomata> restarted solr jetty on solr3 and solr1003 for MaxSem [production]
21:03 <bsitu> Started syncing Wikimedia installation... : Update ArticleFeedbackv5, Echo, PageTriage to master [production]
20:24 <kaldari> synchronized wmf-config/CommonSettings.php 'updating CommonSettings.php for Echo' [production]
20:15 <apergos> rebooting mw1010 after some issues with OOM [production]
19:22 <RobH> new servers are NOT in pybal yet, and thus not getting traffic [production]
19:22 <RobH> just added mw1161-1200 into node lists for eventual service, some may throw sync script errors as I track them down [production]