[Users] Normal?: lots of connections in CLOSE_WAIT

[pf at pfortin.com]

On Mon, 9 Feb 2015 08:34:53 +0100 Albert ARIBAUD wrote:

>Hi Pierre,
>
>Le Sun, 8 Feb 2015 22:11:30 -0500, Pierre Fortin <pf at pfortin.com> a
>écrit :
>
>> Hi,
>> 
>> While looking into other system issues, I noticed a lot of CM TCP
>> connections in CLOSE_WAIT -- 260 at the moment.
>> 
>> Sample output:
>> # lsof
>> cm 2341 9207 pfortin 26u IPv4 364889234 0t0 TCP
>> prf:43695->a23-62-7-48.deploy.static.akamaitechnologies.com:http
>> (CLOSE_WAIT)
>> 
>> Sample sites:
>> prf:43695->a23-62-7-48.deploy.static.akamaitechnologies.com:http
>> prf:34937->a23-0-217-196.deploy.static.akamaitechnologies.com:http
>> prf:56904->a23-0-217-196.deploy.static.akamaitechnologies.com:https
>> prf:50877->a23-62-7-25.deploy.static.akamaitechnologies.com:http
>> prf:38715->server-54-230-19-219.iad12.r.cloudfront.net:https
>> 
>> Only seeing this on this instance of CM -- I run two; the other one is
>> clean.
>>
>> Is this normal?
>
>
>Are the two run on the same machine? And is there a firewall involved?

Yes. No. 
Same binary. 

Finding these was a surprise; but not seeing them on instance #2 which is
accessed via:
  $ ssh pierre at prf
  $ cm &
was puzzling...

Looking more closely, I also see these:
- gdbus      2341  2346    pfortin   26u ...
- gmain      2341  2347    pfortin   26u ...
- dconf      2341  9238    pfortin   26u ...
which are sub-processes(TID) to CM(PID=2341) -- for a total of 364...
Note that when these extras are present, they are all tagged with the same
FD (ie, 26u).  In fact, there are 13[1] copies of the "cm" sample above
-- 1 with no TID and 12 w/different TIDs and everything else, the same,
including FD, TCP port, host...

[1] initially suspected this was based on the accounts; but on this
instance, I have 14 accounts defined, yet only 7 are enabled...  

Plugins loaded (instance 1 is the affected one):

instance 1: Acpi Notifier, Address Keeper, Attach warner, Fancy HTML
Viewer, Libravatar, NewMail, Notification, PDF Viewer, Python

instance 2: Acpi Notifier, Attach warner, Fancy HTML Viewer, NewMail,
Notification, PDF Viewer, PGP/Core, PGP/inline, PGP/MIME, Python,
VCalendar

BTW, the reason I'm looking into "lsof" is because I've been hitting some
strange issues after running at least a week or two without logout/login;
and lsof output grows quite a bit -- from 11MB to ~35MB.  CM entries seem
to be excessive:
  # grep " 2341 " lsof.201502090841 | wc      # instance 1
     5040   50213  685344
  # grep " 2291 " lsof.201502090841 | wc      # instance 2
     4944   49147  667904
for a total of 9984 {cm,gdbus,gmain,dconf} entries in lsof...

As I finish writing this:
  # grep " 2341 " lsof.201502090841 | grep CLOSE_WAIT | wc
      320    3500   56128
  # grep " 2291 " lsof.201502090841 | grep CLOSE_WAIT | wc
        0       0       0

Weird...

>> Cheers,
>> Pierre
>
>Amicalement,

Likewise,
Pierre