Exclude grep itself from ps

This is so simple it’s just great :-)

Solution: use regex in your grep so the grep itself doesn’t show up in the results.

Example:

[vagrant@london kafka]$ ps aux |grep kafka
vagrant 5172 0.8 30.3 3178252 309428 ? Sl 07:00 0:06 java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true -Xloggc:/var/log/kafka/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/var/log/kafka -Dlog4j.configuration=file:/etc/kafka/log4j.properties -cp :/usr/bin/../share/java/kafka/*:/usr/bin/../share/java/confluent-support-metrics/*:/usr/share/java/confluent-support-metrics/* io.confluent.support.metrics.SupportedKafka /vagrant/config/kafka0.properties
vagrant 5824 0.0 0.0 103316 836 pts/0 R+ 07:13 0:00 grep kafka  <<-- Oh no!
[vagrant@london kafka]$


[vagrant@london kafka]$ ps aux |grep [k]afka
root 5172 0.8 29.6 3178252 302472 ? Sl 07:00 0:04 java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true -Xloggc:/var/log/kafka/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/var/log/kafka -Dlog4j.configuration=file:/etc/kafka/log4j.properties -cp :/usr/bin/../share/java/kafka/*:/usr/bin/../share/java/confluent-support-metrics/*:/usr/share/java/confluent-support-metrics/* io.confluent.support.metrics.SupportedKafka /vagrant/config/kafka0.properties
[vagrant@london kafka]$

 

 

Remove duplicate lines while comparing two files

I’ve been quite busy this whole day with a partially complete database dump and wanted to prepare for tomorrow with some ninja bash voodoo shizzle. I’m doing a braindump here because I know I’ll have forgotten this when I wake up tomorrow :-)

The command stated below was the first working example I’ve gotten together, please let me know if you know a neater / better solution!

The situation:

I’ve got two files. The first file contains lines which need to be deleted from the second line (if they exist there) Continue reading “Remove duplicate lines while comparing two files”

Supercharge your CLI bash history search

This is a repost from https://coderwall.com/p/oqtj8w but it’s so handy, I want to share anyway :-)

 

Create ~/.inputrc and fill it with this:

"\e[A": history-search-backward 
"\e[B": history-search-forward 
set show-all-if-ambiguous on 
set completion-ignore-case on

 

This allows you to search through your history using the up and down arrows … i.e. type “cd /” and press the up arrow and you’ll search through everything in your history that starts with “cd /”.

Oracle Service Bus SFTP: BEA-381801 InvalidHostFileException

I have just been working on an OSB SFTP business service, and got this error while testing the service.

While googling, I found out that you need to create a known_hosts file inside your OSB domain. The easiest way of creating a known_hosts file for me, was to use Cygwin and create a ssh (or sftp) connection to the destination.

Steps for the solution:

  • Cygwin: log in to remote system via ssh (or sftp)
  • Copy known_hosts file from ~/.ssh to OSB directory
  • Test business service

Cygwin: login to remote system

Use this command to login to the remote system:

ssh username@hostname

Optionally: Type ‘yes’ when asked if you trust this host

This will create an entry in your known_hosts file in the .ssh folder in your cygwin home directory

This is a (scrambled) contents of the known_hosts file:

joris@jorisworkstation/cygdrive/c/Oracle/Middleware/user_projects/domains/joris_domain/config/osb
$ cat ~/.ssh/known_hosts
10.200.242.65 ssh-rsa <<INSERT A LOT OF GARBLEGARBLE>>

Copy the known_hosts file to OSB

First we need to create the SFTP transports directory in the domain home:

Go to <<domain_home>>/config/osb and create this structure: (Only create the directories in RED)

- <<domain_home>>
- config
- osb
- transports
- sftp

We are now ready to copy the known_hosts file:

In cygwin:

 cp ~/.ssh/known_hosts /cygdrive/c/Oracle/Middleware/user_projects/domains/joris_domain/config/osb/transports/sftp

In Windows: Find known_hosts in your cygwin home folder (in my case: in C:\development\program\cygwin\home\joris.visscher\.ssh\known_hosts) and copy it to the newly created “sftp” folder in your domain home.

Test business service

All steps are now done, we can test the business service!

Additional information:

You will encounter the following error in your logs when this error occurs:

com.sshtools.j2ssh.transport.InvalidHostFileException

####<18-ott-2012 10.57.33 CEST> <Error> <SFTPTransport> <JORISWORKSTATION> <osb_server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <cef9dbcf0859648a:31ba6f:13a730795ec:-8000-000000000000014f> <1350550653493> <BEA-381801> <Si ▒ verificato un errore per l'endpoint com.bea.wli.sb.transports.TransportException: com.sshtools.j2ssh.transport.InvalidHostFileException: Il file known_hosts non esiste o non dispone di autorizzazioni di lettura.
 com.bea.wli.sb.transports.TransportException: com.sshtools.j2ssh.transport.InvalidHostFileException: Il file known_hosts non esiste o non dispone di autorizzazioni di lettura.
 at com.bea.wli.sb.transports.sftp.connector.SFTPTransportProvider.sendMessage(SFTPTransportProvider.java:155)
 at com.bea.wli.sb.transports.sftp.connector.SFTPTransportProvider.sendMessageAsync(SFTPTransportProvider.java:110)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at com.bea.wli.sb.transports.Util$1.invoke(Util.java:83)
 at $Proxy122.sendMessageAsync(Unknown Source)
 at com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageAsync(LoadBalanceFailoverListener.java:148)
 at com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageToServiceAsync(LoadBalanceFailoverListener.java:603)
 at com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageToService(LoadBalanceFailoverListener.java:538)
 at com.bea.wli.sb.transports.TransportManagerImpl.sendMessageToService(TransportManagerImpl.java:566)
 at com.bea.wli.sb.transports.TransportManagerImpl.sendMessageAsync(TransportManagerImpl.java:434)
 at com.bea.wli.sb.test.service.ServiceMessageSender.send0(ServiceMessageSender.java:380)
 at com.bea.wli.sb.test.service.ServiceMessageSender.access$000(ServiceMessageSender.java:79)
 at com.bea.wli.sb.test.service.ServiceMessageSender$1.run(ServiceMessageSender.java:137)
 at com.bea.wli.sb.test.service.ServiceMessageSender$1.run(ServiceMessageSender.java:135)
 at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
 at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
 at com.bea.wli.sb.security.WLSSecurityContextService.runAs(WLSSecurityContextService.java:55)
 at com.bea.wli.sb.test.service.ServiceMessageSender.send(ServiceMessageSender.java:140)
 at com.bea.wli.sb.test.service.ServiceProcessor.invoke(ServiceProcessor.java:454)
 at com.bea.wli.sb.test.TestServiceImpl.invoke(TestServiceImpl.java:172)
 at com.bea.wli.sb.test.client.ejb.TestServiceEJBBean.invoke(TestServiceEJBBean.java:167)
 at com.bea.wli.sb.test.client.ejb.TestService_sqr59p_EOImpl.__WL_invoke(Unknown Source)
 at weblogic.ejb.container.internal.SessionRemoteMethodInvoker.invoke(SessionRemoteMethodInvoker.java:40)
 at com.bea.wli.sb.test.client.ejb.TestService_sqr59p_EOImpl.invoke(Unknown Source)
 at com.bea.wli.sb.test.client.ejb.TestService_sqr59p_EOImpl_WLSkel.invoke(Unknown Source)
 at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:667)
 at weblogic.rmi.cluster.ClusterableServerRef.invoke(ClusterableServerRef.java:230)
 at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:522)
 at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
 at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
 at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:518)
 at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
 at weblogic.work.ExecuteThread.execute(ExecuteThread.java:252)
 at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
 Caused By: com.sshtools.j2ssh.transport.InvalidHostFileException: Il file known_hosts non esiste o non dispone di autorizzazioni di lettura.
at com.bea.wli.sb.transports.sftp.client.KnownHostVerifier.validateKnownHosts(KnownHostVerifier.java:144)
at com.bea.wli.sb.transports.sftp.client.SFTPClient.authenticate(SFTPClient.java:108)
 at com.bea.wli.sb.transports.sftp.connector.SFTPResource.<init>(SFTPResource.java:78)
 at com.bea.wli.sb.transports.sftp.resource.SFTPUtils.createSFTPResource(SFTPUtils.java:170)
 at com.bea.wli.sb.transports.sftp.resource.SFTPConnectionPool.getResource(SFTPConnectionPool.java:104)
 at com.bea.wli.sb.transports.sftp.connector.SFTPTransportProvider.getSFTPResource(SFTPTransportProvider.java:958)
 at com.bea.wli.sb.transports.sftp.connector.SFTPTransportProvider.sendMessage(SFTPTransportProvider.java:145)
 at com.bea.wli.sb.transports.sftp.connector.SFTPTransportProvider.sendMessageAsync(SFTPTransportProvider.java:110)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at com.bea.wli.sb.transports.Util$1.invoke(Util.java:83)
 at $Proxy122.sendMessageAsync(Unknown Source)
 at com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageAsync(LoadBalanceFailoverListener.java:148)
 at com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageToServiceAsync(LoadBalanceFailoverListener.java:603)
 at com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageToService(LoadBalanceFailoverListener.java:538)
 at com.bea.wli.sb.transports.TransportManagerImpl.sendMessageToService(TransportManagerImpl.java:566)
 at com.bea.wli.sb.transports.TransportManagerImpl.sendMessageAsync(TransportManagerImpl.java:434)
 at com.bea.wli.sb.test.service.ServiceMessageSender.send0(ServiceMessageSender.java:380)
 at com.bea.wli.sb.test.service.ServiceMessageSender.access$000(ServiceMessageSender.java:79)
 at com.bea.wli.sb.test.service.ServiceMessageSender$1.run(ServiceMessageSender.java:137)
 at com.bea.wli.sb.test.service.ServiceMessageSender$1.run(ServiceMessageSender.java:135)
 at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
 at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
 at com.bea.wli.sb.security.WLSSecurityContextService.runAs(WLSSecurityContextService.java:55)
 at com.bea.wli.sb.test.service.ServiceMessageSender.send(ServiceMessageSender.java:140)
 at com.bea.wli.sb.test.service.ServiceProcessor.invoke(ServiceProcessor.java:454)
 at com.bea.wli.sb.test.TestServiceImpl.invoke(TestServiceImpl.java:172)
 at com.bea.wli.sb.test.client.ejb.TestServiceEJBBean.invoke(TestServiceEJBBean.java:167)
 at com.bea.wli.sb.test.client.ejb.TestService_sqr59p_EOImpl.__WL_invoke(Unknown Source)
 at weblogic.ejb.container.internal.SessionRemoteMethodInvoker.invoke(SessionRemoteMethodInvoker.java:40)
 at com.bea.wli.sb.test.client.ejb.TestService_sqr59p_EOImpl.invoke(Unknown Source)
 at com.bea.wli.sb.test.client.ejb.TestService_sqr59p_EOImpl_WLSkel.invoke(Unknown Source)
 at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:667)
 at weblogic.rmi.cluster.ClusterableServerRef.invoke(ClusterableServerRef.java:230)
 at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:522)
 at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
 at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
 at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:518)
 at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
 at weblogic.work.ExecuteThread.execute(ExecuteThread.java:252)
 at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)