KeySet & ehcache

KeySet is a view of keys contained in the Map.

In ehcache, I store <id, Map>.

Map oldAuthApInfos = cache.get(id);
SetoldBssids = oldAuthApInfos.keySet();
...
oldBssids.removeAll(newBssids);

This would result in map entries getting removed from cache. By default, ehcache uses memoryStore and basically is a map. The operation on the returned map have direct effect on cache value itself. I used to think the returned map of cache.get() is a serialized copy of the cache value.

Executor Completion Service

Scenario: you want to load the tasks in parallel and then wait for the completion of all the tasks.

//Callable is different from Runnable that it returns result
private final class StringTask extends Callable<String>{
   public String call(){
      //Long operations
      return "Run";
   }
}

ExecutorService threadPool = Executors.newFixedThreadPool(4);
CompletionService<String> pool = new ExecutorCompletionService<String>(threadPool);

for(int i = 0; i < 10; i++){
   pool.submit(new StringTask());
}

//you have the result in the order they are completed and you don’t have to keep a 
//collection of Future
for(int i = 0; i < 10; i++){
   String result = pool.take().get();

   //Compute the result
}
threadPool.shutdown();

See: Java Concurrency: Executors and Thread Pools

MySQL Commands

- Enable/disable MySQL general log (query log)

  % set global general_log = 'ON'   (or 'OFF')

The file is output to hostname.log by default or we can specify file name in my.ini with "log=fileName"

- Turn on/off Foreign Key checks for dropping schema

  % set foreign_key_checks = 1;  (or 0 to turn off)

- Check version

  % status;

Two Linked Structures

- LinkedBlockingQueue
It makes sure all the jobs will be executed sequentially. The take() method retrieves and removes the head of this queue, waiting if necessary until an element becomes available.

- LinkedHashMap
It maintains a doubly-linked list internally and keep the order that the key is inserted.

JMX Service URL

JMX service URL could be in either JNDI form or encoded form.

JNDI form: the RMI stub is acquired from external such as RMI registry

Encoded form: the generated stub is encoded and attached in service url itself. So no RMI registry is required.

The JMX service URL has the following syntax:

service:jmx:rmi://[host[:port]][urlPath]

Although host and port may be included, they are ignored by the RMI protocol. If urlPath is specified, it gives the Java Naming and Directory Interface (JNDI) location of an RMI stub (typically a location within an RMI registry) in the form

/jndi/jndiName

For example, the URL

service:jmx:rmi://myhost/jndi/rmi://myhost:1099/myhost/myjmxconnector

specifies an RMI stub at the location

rmi://myhost:1099/myhost/myjmxconnector

which is an RMI registry running at location myhost/myjmxconnector on port 1099 of host myhost.

Alternatively, if urlPath is omitted from the service URL, the JMX connector server will generate a client URL containing the actual RMI stub embedded within it in encoded and serialized form. For example, the service URL

service:jmx:rmi://localhost

will generate a client URL of the form

service:jmx:rmi://localhost/stub/rmiStub

where rmiStub is an encoded and serialized representation of the RMI stub itself.

    // Get the platform MBeanServer
    MBeanServer mbeanServer = ManagementFactory.getPlatformMBeanServer();

    // Attach a JMXConnectorServer to the platform MBeanServer
    JMXServiceURL jmxServiceURL = new JMXServiceURL("service:jmx:rmi:///");
    JMXConnectorServer connectorServer = 
            JMXConnectorServerFactory.newJMXConnectorServer(jmxServiceURL, null, mbeanServer);
    connectorServer.start();

    // Refresh the JMXServiceURL after the JMXConnectorServer has started
    // it is different with the original one with the stub object attached.
    jmxServiceURL = connectorServer.getAddress();

See:
JMX Service URL example
JMX remoting API
JMX Service URLs

Notify and Wait

Two things to be noticed:
1) notify() and wait() need to be used inside a synchronized block. They are used for release/acquire object lock.

2) In getMessage() method, notify() is called at first beginning. It doesn't actually release the lock atm, as stated in API "Wakes up a single thread that is waiting on this object's monitor...The awakened thread will not be able to proceed until the current thread relinquishes the lock on this object."

import java.util.Vector; 
 
class Producer extends Thread { 
    static final int MAXQUEUE = 5; 
    private Vector messages = new Vector(); 
  
    public void run() { 
        try { 
            while ( true ) { 
                putMessage(); 
                sleep( 1000 ); 
            } 
        }  
        catch( InterruptedException e ) { } 
    } 
 
    private synchronized void putMessage() 
        throws InterruptedException { 
        
        while ( messages.size() == MAXQUEUE ) 
            wait(); 
        messages.addElement( new java.util.Date().toString() ); 
        notify(); 
    } 
 
    // Called by Consumer 
    public synchronized String getMessage() 
        throws InterruptedException { 
        notify(); 
        while ( messages.size() == 0 ) 
            wait(); 
        String message = (String)messages.firstElement(); 
        messages.removeElement( message ); 
        return message; 
    } 
} 
 
class Consumer extends Thread { 
    Producer producer; 
     
    Consumer(Producer p) { 
        producer = p; 
    } 
  
    public void run() { 
        try { 
            while ( true ) { 
                String message = producer.getMessage(); 
                System.out.println("Got message: " + message); 
                sleep( 2000 ); 
            } 
        }  
        catch( InterruptedException e ) { } 
    } 
  
    public static void main(String args[]) { 
        Producer producer = new Producer(); 
        producer.start(); 
        new Consumer( producer ).start(); 
    } 
} 

See: Exploring Java: Threads

MySQL InnoDB failed

The InnoDB storage engine failed to start so all tables were created as MyISAM tables with MySQL’s strange behaviour. As a result, all FK constraints were removed and crashed the server thereafter. (Tip: use "show create table xxx" to check the created table schema.) It seems that during installation, MySQL is not able to create innodb temporary file as shown in error logged in the file mysql\data\.err.

101006 14:36:46 InnoDB: Error: unable to create temporary file; errno: 2
101006 14:36:46 [ERROR] Plugin 'InnoDB' init function returned error.
101006 14:36:46 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.

The reason is that when innodb starts the tmpdir defined in my.ini should have existed.

Timer Daemon thread

public Timer(boolean isDaemon)

Creates a new timer whose associated thread may be specified to run as a daemon (background).A daemon thread exit automatically when JVM exits. If it is not a daemon (foreground), timer.cancel() need to called explicitly to make the thread exit.

When a JVM shutdown hook is registered, it waits for all non-daemon threads exit with thread.join(). I run into this issue when there is a non-daemon timer thread and block the JVM from exiting.

Dead Lock

The server run over the weekend, on Monday morning it appeared not working any more as the log shown. My first impression is the process has exited, but why there is no error reported in log. I checked Task Manager, there were two java processes Id, then I looked at Process Explorer, from tree view, it shows one is parent, the other is child which is the one I am interested. So the process is still alive, why doesn't it work then? I used JDK's tool 'jstack' to print out all the thread stacks associate with this child process Id. At the end of the output, it displayed a dead lock have been detected.

Found one Java-level deadlock:
=============================
"Timer-2":
waiting to lock monitor 0x4734334c (object 0x079cd3b8, a com.xyz.PunaAdapter), which is held by "Timer-1"
"Timer-1":
waiting to lock monitor 0x475a397c (object 0x0a946f60, a com.xyz.SgeSessionManager),which is held by "Timer-2"

I wonder for a while

Criteria crit = getSession().createCriteria(ApSummary.class)
.add(Restrictions.eq("controller.id", controllerId));

This "controller.id" surprises me a bit since when we do a join in Criteria query, generally we have to use createAlias() such as:

Criteria crit = getSession().createCriteria(ApSummary.class)
.createAlias("controller", "hwc").add(Restrictions.eq("hwc.id", controllerId));

The trick here is that controllre.id would be SMARTLY interpreted as column "controllerId" in ApSummary without doing the extra join!

A lot of optimism with a little pessimism

org.springframework.orm.hibernate3.HibernateOptimisticLockingFailureException: Object of class [xyz.WlanServiceTemplate] with identifier [17]: optimistic locking failed; nested exception is org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect):

This StaleObjectStateException is quiet common when optimistic locking strategy (FIRST WIN) is applied in persistence. Generally it indicates the caller side has a out-of-date copy than the one in DB. It prevents the lost-update issue, which is OK for UI since the use could reload and retry. But for backend, this StaleObjectStateException may be annoying because LAST WIN may be acceptable. One straightforward but cumbersome approach is to catch the exception, rollback and retry again until succeed.

A much better approach is to apply pessimistic lock (session.get(id, LockMode.UPGRADE) in such scenario, so called "a lot of optimism with a little pessimism" which neatly avoid the need to EVER catch or recover from StaleObjectExceptions!

Custom BIT functions in HQL

public class MySQLDialect extends MySQLInnoDBDialect {

    public MySQLDialect() {
        super();
        registerFunction("bit_or", new StandardSQLFunction("BIT_OR", Hibernate.INTEGER));
        registerFunction("bitwise_and", new VarArgsSQLFunction(Hibernate.INTEGER, "", "&", ""));
        registerFunction("bitwise_or", new VarArgsSQLFunction(Hibernate.INTEGER, "", "|", ""));
    }
}
 
 String queryString = "select bit_or(cc.protocol) as protocolBitmask "
     + "from CtlCountry cc, ApHardwareTypeProfile atp, ApProfileRadio radio "
     + "where cc.platformName=atp.ctlPlatformName "
     + "and atp.profileType=:profileType "
     + "and atp.profileType=radio.profile "
     + "and cc.countryCode=:country " 
     + "and radio.radioIndex=:radioIndex "
     + "and bitwise_and(cc.protocol, radio.radioCap) = cc.protocol";

MySQL's BIT_OR(), BIT_AND() operators are like COUNT(), AVG(), they apply bit operations on all matched rows. Bitwise operators &, | apply on expression.

A strange "java.lang.IllegalStateException: No data type for node: org.hibernate.hql.ast.tree.MethodNode" error occurred when I used upper case function name "BIT_OR" in HQL. After I changed it to use lower case "bit_or", it works.

Generic Method

    public static <T extends Template<T>> T detailsCopy(T template, boolean snapshot);
    public static <T extends Template> Set<T> detailsCopy(Set<T> templateSet, boolean snapshot);
    public void draw(List<? extends Shape> shape);

EventQueue & Session timeout tracking

Swing's single-threaded model make UI updates run in Event Dispatch thread. Those actionPerformed listener method of UI components already runs in Event dispatch thread. The EventQueue.invokeLater(Runnable), EventQueue.invokeAndWait(Runnable) cause runnable to have its run method called in the dispatch thread of the system EventQueue. The cases occur for long lasting task and run in separate thread (SwingWork provides convenient methods for it).

SwingUtilites is a wrapper class of EventQueue for old java code.

We could provide our custom EventQueue for System Event Queue, which contains events processed by event dispatch thread. It looks like Swing launch a new AWT-EventQueue- thread when a new EventQueue is set.

    Toolkit.getDefaultToolkit().getSystemEventQueue().push(new NSEventQueue());

For custom EventQueue, we can override its dispatchEvent() method and record the keepAlive timestamp for mouse and keyboard activities. It could be used for session timeout tracking.

See: Swing threading and Dispatch thread

Secure Acess in web.xml

 <security-constraint>
        <web-resource-collection>
            <web-resource-name>Restricted Access</web-resource-name>
            <url-pattern>/jsp/*</url-pattern>
            <http-method>POST</http-method>
            <http-method>GET</http-method>
        </web-resource-collection>
        <auth-constraint>
            <role-name>NetSightAdministrator</role-name>
            <role-name>NetSightUser</role-name>
        </auth-constraint>
        <user-data-constraint>
            <transport-guarantee>CONFIDENTIAL</transport-guarantee>
        </user-data-constraint>
    </security-constraint>
     
    <security-constraint>
        <web-resource-collection>
            <web-resource-name>Secured Access</web-resource-name>
            <url-pattern>/sessionDetails</url-pattern>
            <http-method>GET</http-method>
        </web-resource-collection>
        <user-data-constraint>
            <transport-guarantee>CONFIDENTIAL</transport-guarantee>
        </user-data-constraint>
    </security-constraint>
    
    <login-config>
        <auth-method>FORM</auth-method>
        <form-login-config>
            <form-login-page>/login.jsp</form-login-page>
            <form-error-page>/failure.jsp</form-error-page>
        </form-login-config>
    </login-config>

The 1st "Restricted Access" requires login and https access. The 2nd "Secured Access" doesn't have auth-constraint tag and only requires https access. (*The empty auth-constraint tag has different meaning - to exclud access). The user-data-constraint tag to specify http/https access, value NONE or CONFIDENTIAL.

See: Configuring security on a resource

Super class Serialization

The superclasses of your serialized classes need not be Serializable. However, those superclass fields won’t be saved/restored. The fields will be restored to whatever you would get running the non-arg constructor.

http://mindprod.com/jgloss/serialization.html

Groovy

Groovy: an agile dynamic language for the Java Platform

Closure: Groovy's way of providing transparent callback, used for two particular areas: collection iteration and resources handling.
{ [closureArguments->] statements }

(1..10).each{counter -> log += counter} 

// method inject(object, closure)
[1, 2, 3].inject(0) {count, item -> count + item}

// multiple parameter
map = ['a':1, 'b':2]
map.each{ key, value -> map[key] = value * 2}

// call closure
def adder = {x, y -> return x + y}
assert adder(4, 3) == 7
assert adder.call{2, 6} == 8

// curried closure
def adder = {x, y -> return x + y}
def addone = adder.curry(1)
assert addone(5) == 6

Think of the arrow as an indication that parameters are passed from the method on the left into the closure body on the right.

The real power for currying comes when the closure's parameters are themselves closures. Using composition, two or more simple closures can be combined to produce a more elaborate one.

GroovyBeans: when no visibility modifier attached to field declaration, a property is generated. i.e., a private field and two public accessor methods (overridable). (Note that property!=field). When final keyword is used, the property will only be readable, not setter method created.

class MyBean implements Serializable {
    String myprop;   // default visibility
}

Multimethods: Groovy's mechanics of method lookup take the dynamic type of method arguments into account, whereas Java relies on the static type. (Note: The dynamic binding of Java (Polymorphism) is for caller object, not method argument). e.g., Object's default equals method can be overridden as:

class Equalizer {
    boolean equals(Equalizer e) {
        return true
    }
}

GPaths spread-dot operator: list*.member where *. is called spread-dot operator and member can be a field access, a property access, or a method call.

list*.member  <==> list.collect{ item -> item?.member }
list.property <==> list.collect{ item -> item?.property }  // abbre of special case for property

Groovy-Eclipse plugin installation

I have been trying to install Groovy Eclipse plugin using update sites listed in Groovy, but always failed with errors such as "No repository found containing:org.codehaus.groovy/osgi.bundle/1.5.7.20081120_2330" or missing dependency etc.

Finally, I downloaded the snapshot (archived zip) to local and install it successfully.
# Help -> Install new software -> Add...
# Select the location of the zip you just downloaded
# Install as you would from the regular update site

JNDI in JBoss

There are three levels of naming scope in JBoss:

- Name underjava:comp, only available to the application component such as EJB Bean1, Web application web1 etc. java:comp/env refers to as enterprise naming context (ENC)

- Names under java:, visible only within the JBoss server and not to remote clients

- Any other names, global visible, provided that the context or object support serialization.

With JNDI reference, it allows us to bind non-serializable object to JNDI. It actually only bind object reference to it and retrieve the object from an ObjectFactory based on the reference. Spring's BeanFactory is bound to JNDI in this way. The limitation is that the object is only available in the same JVM. Even a backend service is serializable, it doesn't make sense to bind it to JNDI directly, because the implementation class gets loaded to client side and unserialized.

A code snippet from JBoss's NonSerializableFactory

public static synchronized void rebind(Context ctx, String key, Object target)
       throws NamingException
   {
       rebind(key, target);
       String className = target.getClass().getName();
       String factory = (org.jboss.util.naming.NonSerializableFactory.class).getName();
       StringRefAddr addr = new StringRefAddr("nns", key);
       Reference memoryRef = new Reference(className, addr, factory, null);
       ctx.rebind(key, memoryRef);
   }

Drop database from DOS batch script

mysql -uroot -pabc123 -e "drop database netsight" >nul 2>1&

The option -e to run execute a command. --help (-?) to display help information.

Redirect to null to keep quiet. Otherwise, it may display an error "ERROR 1010 (HY000) at line 1: Error dropping database (can't rmdir '.\netsight', errno: 41)". Since when dropping a database, it will try to delete folder mysql/data/netsight, it fails if the folder contains some non-db files.

Insert for multiple records

insert into ce_antennaSocket(platformName, socketID, socketName, radioBandID) values
('AP2600', '0', 'Left Antenna Type', 0), 
('AP2600', '1', 'Right Antenna Type', 0), 
('AP2650', '0', 'Top Left Antenna Type', 0), 
('AP2650', '1', 'Bottom Left Antenna Type', 0), 
('AP2650', '2', 'Top Right Antenna Type', 0), 
('AP4102', '1', 'Right Antenna Type', 1);

It has much much better performance than split into multiple insert statements. To insert 15k records, it took less than 1 second. Otherwise, it took 8 minutes.

Class vs. Classloader getResouceAsStream

In bk.sar -
conf/bkConfigSchema.sql
com.xyz.BkWirelessMgr

For classloader.getResouceAsStream(), all names are absolute, no leading "/". For class.getResourceAsStream(), it has relative and absolute path. But relative means starting from the class's package, i.e., without the leading "/", it will search com.xyz/conf/bkConfigSchema.sql, not found and return null.

Then in BkWirelessMgr class, to load the sql,

getClass().getResourceAsStream("/conf/bkConfigSchema.sql");
getClass().getClassLoader().getResourceAsStream("conf/bkConfigSchema.sql");

Note that "/" is used here instead of File.Separator which doesn't work.

ParentAppContext

The bk-core.spring is deployed by Spring Deployer and its applicatonContext could be parent context of the webApplicatonContext (loaded by ContextLoaderListener) in bk-config-engine.war to share the beans in bk-core. To configure this:

1. In bk-core's jboss-spring.xml

BeanFactory=(java:/bkCore);

Then the beanFactory will be bound to JNDI in unserialized form with the given name by Spring deployer.

2. In bk-config-engine.war's web.xml

    parentContextKey
    bkCoreAppContext

3. Add beanRefContext.xml in bk-config-engine.war (located in classpath)

<jee:jndi-lookup id="bkCoreAppContext" jndi-name="java:/bkCore"/>

Choose XML Parser

System.setProperty("javax.xml.parsers.DocumentBuilderFactory", "com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl");
System.setProperty("javax.xml.parsers.SAXParserFactory", "com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl");

Hibernate Native SQL

Hiberante HQL/Criteria doesn't support the query for Map collection with dereferenced key object. AddEntity() method will convert the result into domain object, otherwise, it would be an object array.

    String sqlStr = "select t.* from ce_template t, ce_apProfileTemplate_wlanServiceTemplate aw "
         + "where t.id=aw.apProfileTemplateId and t.templateType='APPROFILE' and t.oldId=0 "
         + "and aw.wlanServiceTemplateId=:id";
    Query query = getSession().createSQLQuery(sqlStr)
          .addEntity("t", Template.class)
          .setInteger("id", t.getId());

烟霞闲骨格，泉石野生涯。

探春秋爽斋内对联

Formula in Map

This has the drawback that the embedded component class (Image) doesn't have the property imageName which is the map key. It may be undesirable.

In Hibernate 3.x, formula comes as a rescue here.

[Reference] Hibernate: how to map a collection of embedded components keyed by one of the component's properties?

n+1 select & fetch strategy

The n+1 select is for return *LIST* of objects, and the object contains collection. Then we can use 'subselect' fetch strategy to reduce the number of queries. For example, Template (1-->*) DeployedController, with default fetch strategy 'select', to return a list of template it requires 1 + n selects.

select * from template;
select * from deployedController where templateId=?
select * from deployedController where templateId=?
...

With 'subselect', it becomes only 2 queries -

select * from template;
select * from deployedController where templateId in (select id from template)

This is for lazy loading, the 2nd query is run when the 1st time the deployedController collections is accessed. For eager loading, change fetch strategy to 'join', then only one query (left outer join) is used but the resultset contains duplicate or null values.

The subselect option is currently only available for collection, not for *-->1 assocation.

Eager loading:
1. lazy="false", stands for fetch="select", follow by an immediate second select
2. fetch="join"
Lazy Loading:
1. fetch="subselect"
2. batch-size="N"

Common Hibernate Exceptions

1. a different object with the same identifier value was already associated with the session
In session cache, a persistent object exists uniquely. To avoid this exception, use merge() instead of saveOrUpdate().

One example, many-to-one between authRadius->radius, and I have authRadiusSet[authRadius1(radius1), authRadius2(radius2)], here radius1 and radius2 actually have same database identifier but different objects. When I tried to save authRadiusSet, I got this exception.

2. Duplicate entry '1' for key 2
The object tree for cascade save/update contains two objects with same object id (java object).

3. A collection with cascade="all-delete-orphan" was no longer referenced by the owning entity instance
What is happening here is that Hibernate requires complete ownership of the preferences collection in the User object. If you simply set it to a new object as in the above code sample, Hibernate is unable to track changes to that collection and thus has no idea how to apply the cascading persistence to your objects! So either user.setPreferences(new HashSet()) or user.setPreferences(null) will cause such error. This is also a reason to provide helper methods such as addPreference() and removePreference() to avoid setPreferences is called (made private). See ...

4. Found shared references to a collection org.hibernate.HibernateException
It throws when session flush. It is due to one collection referenced by two entities. To relove it, use merge().

5. TransientObjectException
The object graph to be persisted cannot contains any transient objects (id != 0) without cascade defined. Also note that, if it contains detached object (id >0) and no cascade defined, the detached object will be deleted upon persisting.

6. ObjectDeletedException
Deleted object would be re-saved by cascade. It occurs when you move one object from one collection to the other and you have 'delete-orphan'. There is not good solution.
https://forum.hibernate.org/viewtopic.php?t=981282

enum != Enum

All enumerations automatically contain two predefined methods: values( ) and valueOf( ).

1. public static enum-type[ ] values( )
2. public static enum-type valueOf(String str)

These two methods are not in Enum API.

enum Week {
  Monday, Tuesday, Wednesday, Thursday, Friday, Saturaday, Sunday
}
  
Week allWeek[] = Week.values();

Bag vs. List

public class FilterGroup {
    private List<Fitler> filters = new ArrayList<Fitler>();
    ...
}

I mapped List as above but got bizarre result. Although there are only 8 items in the list, it returned a list with over 1000 item and most of them are null. It turns out this is because the column "orderPos" used for list index contains a big number 1025+.

I do want a sorted list. So I change to use bag mapping with order-by. It works well. In Hibernate document, it said bag is mapped for Collection. Internally it is implemented as ArrayList but ignores index (Bag is different from set in that it allows duplicates). So I can map it to List as well and add a order-by to sort it in database level. (set, map, bag all can have order-by)

Also, note that we need "delete-orphan" cascade for collection. So when we do a update on FilterGroup, the filters removed from the collection will be deleted from database automatically. Otherwise, it will still remain.

Session.get() return proxy?

Surprising, the object returned by session.get() is a proxy rather than the real object. What is going on?

Hibernate API said Session.load(id) return a proxy and Session.get(id) return a persistent object. But actually get() may return proxy if previously it is lazily loaded in the same session, e.g., as a property of other object. A proxy is a proxy even you have called its methods.

Typecasting and instanceof work well for proxy object, however, there is a pitfall, for super-class or interface, it will not give you the implementation class. e.g., it is instanceof Template, but not instanceof VnsTemplate.

To ensure we always get the real persistent object, I implement read() in Dao as follows:

public T read(int id) {
    T obj = (T)getSession().get(entityClass, id);  

    if (obj instanceof HibernateProxy) {
        return(T) ((HibernateProxy)obj).getHibernateLazyInitializer().getImplementation();
    }
  
    return obj;
}

Alternatively, HibernateProxyHelper.getClassWithoutInitializingProxy(o) could be used to get class info.

Note that we couldn't use session.evict() to detach old proxy object and call get() to retrieve the real object. The old proxy object will still be referenced by other object, and access to its member may throw LazyInitializationException even inside the transaction since it has been detached.