Judge Mental

Uses of wildcard types in Java

2012-01-03T18:24:00.000-08:00

Many of the Java coding questions I get from my colleagues come down to confusion about wildcard types in Java generics. I'll be editing this post on an ongoing basis to catalog all the ways I have productively used these beasts.

Uses of wildcard types

Compile-time prevention of incremental modifications to a collection. Because the put() and add() methods of most collection abstractions receive an argument of the element type, the compiler will prevent you from even attempting to call these methods on a variable of type Collection< ? extends T > for any type or type parameter T. Unfortunately, you can still call Collection.clear() with no error, so the collection is not completely statically protected from modification.
Compile-time enforcement of write-only collections. The converse of the above is that the get() methods don't work on a variable of type Collection< ? super T > unless the receiving type is Object, which is usually the wrong thing to do anyway. However you can happily put() and add() Ts to such a collection (and of course you can call clear()).
Broader applicability of parametric library methods. If you have, for example, a utility forAll() which iterates over a collection of elements of type T (and does not use that collection in any other way), the argument type should be Iterable< ? extends T >, and not, say, List< T >, and you should probably have an Iterator< ? extends T > overload as well. If T were CharSequence, for example, you could call forAll( Arrays.asList( "a", "b", "c" ) ) or forAll( new HashSet< StringBuilder >() ) without needing extra copies of the utility logic, casts, or pointless reference copies. If your method requires a more complex nested type, you should use as many wildcards as are allowed, as in Map< ? extends K, ? extends Iterable< ? extends V > >.
Broader utility of parametric library methods. Continuing the above example: because Java has no easy way to do a higher-order method like map() or filter() on collections, you will likely be writing loops in utility methods. Say we are writing a method that iterates a collection, looking for some property of the elements, and produces a new list with only the elements of the original list that had the property in question. It is better to declare the interface like
```
public static List< T > filter( Iterable< ? extends T > it );
```
than
```
public static List< T > filter( Iterable< T > it );
```
Why? Say we have a s of type Set< ? extends Person >. In the latter case, Java will infer the return type of filter( s ) to be List< ? extends Person >, which can't be used directly by some other method needing a List< Person >; not all APIs are considerate enough to follow the protocol in use number 3, above, which leaves them subject to quirk number 3, below. Perhaps surprisingly, in the former case, Java infers a return type of List< Person > for the call filter( s ).
More flexible class implementations. Generic collaborators (such as Comparators) to whom you only supply Ts should be typed using ? super T, and collaborators from whom you only get Ts should be typed using ? extends T. Note that these collaborators might very well be invariant in their type parameters (that is, there may be methods for consuming and producing Ts available); but if you only use one sort or the other, you should still use wildcards to allow covariance or contravariance, whichever is compatible.

Making up for Java's lame type inference. The following innocuous-looking code will not compile as is:


List< List< String > > llstr = Arrays.asList( new ArrayList< String >() );

The reason is that the compiler infers the type List< ArrayList< String > > for the right-hand side of the assignment, which is not assignment-compatible with List< List< String > > (see quirk number 2, below). Any of the following edits will compile, however.


// eschew type inference
List< List< String > > llstr = Arrays.< List< String > >asList( new ArrayList< String >() );
// use a cast (horrors!)
List< List< String > > llstr = Arrays.asList( ( < List< String > > )new ArrayList< String >() );
// use a wildcard in the variable type
List< ? extends List< String > > llstr = Arrays.asList( new ArrayList< String >() );

Potential use of wildcard types, if only the damn language would allow it

Anonymous intersection types. Suppose you want to act on a collection of objects that implement the two interfaces I and J (the technique works for any finite number of interfaces and optionally 1 class). Your method signature could look like
```
public void doSomething( List< ? extends I & J > lij ); // won't compile
```
except that constraints are only allowed where type parameters are declared (i.e., class C< T extends I & J > {}, interface E< T extends I & J > {}, < T extends I & J > void f() {}. You are thus reduced to giving a name to the intersection type, as in
```
public < IJ extends I & J > void doSomething( List< IJ > lij );
```
An advantage of using a name instead of a wildcard is that this technique works directly with a single argument of both interfaces:
```
public < IJ extends I & J > void doSomething( IJ ij ) {
    // ...
    ij.iMethod();
    ij.jMethod();
    iConsumer( ij );
    jConsumer( ij );
    // ...
}
```

Quirks of wildcard types

You can't construct an instance of a wildcard type using new. You can, however, use a factory method to accomplish the same thing, which makes me wonder why the designers of Java didn't just make new work. Given
```
class C< T > {
    public C() {}
    public static < T > C< T > make() {
        return new C< T >();
    }
}
```
the following lines (using type inference) are ok:
```
C< ? extends T > ct = C.make();
C< ? super T > ct = C.make();
```
but the following won't compile ("wildcard not allowed at this location"):
```
C< ? extends T > ct = new C< ? extends T >();
C< ? super T > ct = new C< ? super T >();
C< ? super T > ct = C.< ? super T >make();
```
Generic types are invariant in their type parameters. In simpler terms, a C< Employee > is not a C< Person >, even if it should be by all rights (for example, if there are no methods consuming the parameter type T in C< T >). The opposite relationship also does not hold, even if it should by all rights. Instead, you get to make either relationship hold at the point where such objects are used, usually by using wildcard types. A C< Employee > is a C< ? extends Person >, and a C< Person > is a C< ? super Employee >.
Wildcard types are supertypes. A C< Person > is a C< ? extends Person > and also a C< ? super Person >. No other relationships hold between the three types. It is for this reason that I advocate interfaces like uses 3 and 4, above; a method declared to expect a List< Person > is just not going to accept a List< ? extends Person > or a List< Employee >, even if it only consumed elements from the input as Persons.

Requirements for using a Hadoop combiner

2011-08-05T12:21:00.000-07:00

One way to reduce I/O for large Hadoop jobs, especially those whose mappers produce many records with relatively fewer distinct keys, is to introduce a combiner phase between the mapper and reducer. I have not seen a simple diagram explaining what the types should be for the combiner and what properties the algorithm must exhibit, so here goes. If your mapper extends Mapper< K1, V1, K2, V2 > and your reducer extends Reducer< K2, V2, K3, V3 >, then the combiner must be an extension of Reducer< K2, V2, K2, V2 >, and the following diagram must commute:

The triangle in the center of the diagram represents distributivity of the combiner function (i.e., reduce(k, combine(k, M)) = reduce(k, combine(k, ∪_icombine(k,σ_i(M))) for any partition σ = {σ_i | i ∈ I} of M), because Hadoop does not guarantee how many times it will combine intermediate outputs, if at all.

A common pattern is to use the same function for combiner and reducer, but for this pattern to work, we must have K2 = K3 and V2 = V3 (and of course, the reducer itself must be distributive).

Escaping property placeholders in Spring XML config

2011-06-16T22:05:00.000-07:00

The problem

You might have encountered the awkward situation in which you are

using Spring and XML config
substituting properties into that config via some PropertyPlaceholderConfigurator
needing to set some value in the config to a literal string of the form "${identifier}"

By default, any string of the form in 3. above is a placeholder, and if you have no value for that placeholder, you get an exception. Spring JIRA issue SPR-4953, which recognizes the fact that there is no simple escaping syntax for placeholders, is still open as of this writing.

A snippet such as the following will cause the exception if there is no value available to substitute for the variable customerName, or actually substitute a value for it if it is available. Neither result is desirable in our scenario; we want the "${customerName}" to remain intact when it is injected into our bean.

<bean id="aTrickyBean" class="org.anic.veggies.AreGoodForYou">
    <constructor-arg name="expression" value="Hello, ${customerName}!"/>
</bean>

Most workarounds I have seen are unsatisfactory. You can use a customized placeholder configurator that sets its delimiter characters to something other than the default, for example, which would mean you would have to change the look of all the actual (unescaped) placeholders just to support the ones you want escaped.

The workaround

However, in Spring 3.x, you can work around this issue in a much more simple way using the following trick with SpEL:

<bean id="aTrickyBean" class="org.anic.veggies.AreGoodForYou">
    <constructor-arg name="expression" value="#{ 'Hello, $' + '{customerName}!' }"/>
</bean>

Note that in order for this trick to work it is vital that the '$' and the '{' be physically separated (in this case, on either side of a string concatenation).

Random notes on Hadoop

2010-11-11T00:41:00.000-08:00

I am talking about Hadoop 0.20 using a custom jar, not streaming or Hive or Pig.

Make sure your distribution has the MAPREDUCE-1182 patch.
Make sure you change the default setting of dfs.datanode.max.xcievers to something very large, like 4096. And yes, the property name is misspelled. In 0.22/0.23 the property will be called dfs.datanode.max.transfer.threads.
If you've declared your key type to be T, you can't write an S, even if S is a subclass of T.

There are several ways to get your dependencies visible to Hadoop tasks and tools, and they are all clunky.
- You can bundle them all into the job jar in a folder called lib, although doing so does not make it possible to have custom input formats, output formats, mappers, and reducers in separate jars.
- You can use the -libjars argument, but if you ever have to load a class via reflection (i.e., using Class.forName), you have to use the Thread.currentThread().getContextClassLoader() rather than the default. Also you might run into HADOOP-6103.
- You can use DistributedCache.addFileToClassPath, but you have to be sure to put the file on HDFS and refer to it by its absolute pathname without a scheme or authority, and these files are only available to the tasks, not the tool/job.

DistributedCache is just plain wonky. You must
1. put your file on hdfs somehow
2. call DistributedCache.addCacheFile(), using the full URI with "hdfs:///"
3. in the mapper/reducer, use java.io.* APIs to access the files represented by the paths in DistributedCache.getLocalCacheFiles(). Incredibly, the "typical usage" example in the javadocs for DistributedCache just completely elides this crucial bit. If you try to use FileSystem.get().open(), you'll get a cryptic error message with a filename that looks like it's been mangled.
  I can't find a programmatic mapping between files added via addCacheFile() and paths retrieved by getLocalCacheFiles(), although there may be some support for sharing the name or specifying it with the "hdfs://source#target" syntax. None of this API is well-documented.
You can't substitute Hadoop counters for actual aggregation in your reducer, tempting as that might be. Counters will differ from run to run, even against identical inputs, because of things that vary like speculative execution and task failures.

6 10 Things I Hate About Java (or, Scala is the Way and the Light)

2010-10-24T23:50:00.000-07:00

I've been working with Java quite extensively for about 4 years now, and it has been enjoyable for the most part. Garbage collection, the JVM, generics, anonymous classes, and superb IDE support have made my life much easier.

But a few things make me gnash my teeth on a daily basis, and it's funny, but none of them are issues in another JVM language in which I have been dabbling, Scala. It seems the developers of that language felt my pain as well.

Miserable type inference. Apparently some of the problems with it are being addressed in project coin for Java 7. The blue portions of the following code are, to any sane programmer, maddeningly superfluous, but nevertheless strictly required in Java until at least mid-2011:
List< Integer > li1 = new ArrayList< Integer >(); List< Integer > li2 = Arrays.asList( 1, 2, 3 ); o.processListOfInteger( Arrays.< Integer >asList() );
Needless to say, equivalent initializations in Scala require no such redundant information.
Generic invariance. An example from last week: I'm working on an implementation FrazzleExecutorService of java.util.concurrent.ScheduledExecutorService and a refinement FrazzleFuture< T > of java.util.ScheduledFuture< T >. Covariant subtyping lets me get away with returning a FrazzleFuture< T > from a method like FrazzleExecutorService.submit() without violating the contract. But I can't return List< FrazzleFuture< T > > from FrazzleExecutorService.invokeAll() because (a) ScheduledExecutorService would have had to declare the return type to be List< ? extends ScheduledFuture< T > >; (b) the returned list should have been immutable anyway; (c) generic types like List< T > are invariant in their parameters. In Scala, there is a separate mutable and immutable collections hierarchy, and at least in the immutable one, S <: T implies List[ S ] <: List[ T ], because List is declared covariant in its parameter.
Collections can't be tersely initialized. Part of the blame is the Collections framework; part of it is the goddamn language. The following code illustrates, with green text indicating typical verbosity:
List< Integer > li1 = new ArrayList< Integer >( Arrays.asList( 1, 2, 3 ) ); @SuppressWarnings( "serial" ) Map< String, String > mss1 = new HashMap< String, String >() { { put( "foo", "bar" ); put( "this", "sucks" ); } };
Turns out the collections literals are also not supported in Scala; you have to type List( 1, 2, 3 ) or Map( "foo" -> "bar", "this" -> "rocks" ). Excuse me if I mock Java incessantly at this point. Collection improvements have been postponed until Java 8, scheduled for mid-2012.
Higher-order programming is absurdly verbose. Rather than give code samples, I'll just refer you to these guys and let you see for yourself how even a library can't save you from massive boilerplate for the simplest things. And Scala? Lambda expressions are built-in as syntactic sugar for functional objects, making higher-order code simple, readable, and terse.

Modeling variant types is awkward. You have to choose from among many bad options:

Representation	Interrogation
one sparsely-populated class (S + T modeled as S × T)	`if`-ladders based on comparing to `null` (see 8)
S × T and an enum of type labels	switch + casting
a hierarchy	a bunch of `isS()` and `isT()` methods and casting
a hierarchy	various casting attempts wrapped with `ClassCastException` catch blocks (ok, that's not really an option, but I get that as an answer in interviews sometimes)
a hierarchy	`if`-ladders based on `instanceof` and casting
a hierarchy	polymorphic decomposition and the inevitable bloated APIs at the base class that result
a hierarchy that includes Visitor	painfully verbose visitors (see 4 and 10)
a hierarchy of `Throwable`s	throw and various `catch` blocks, which I suspect compiles to the same thing as the instanceof approach, only more expensive (but actually requires the least code!)

Scala has case classes and pattern matching built in.

No tuples. One ends up either creating Pair< S, T > or dozens of throwaway classes with "And" in the name, like CountAndElapsed. Scala has tuples, although I feel like they kind of screwed up by not going the full ML and making multi-argument functions/methods be the same as single-argument functions/methods over tuples. So to call a 2-arg function f with a pair p = ( p1, p2 ), you can either call f( p1, p2 ) or f.tupled( p ). There must be some deep reason for making the distinction.
No mixins. If you need stuff from 2 abstract classes, you will be copying, or aggregating (with loads of monkey delegation boilerplate) at least one of the two.
Null. The following code should illustrate:
private static Doohickey getDoohickey( Thingamajigger t ) { Whatsit w; Foobar f; if ( null == t ) return null; else if ( null == ( w = t.getWhatsit() ) ) return null; else if ( null == ( f = w.getFoobar() ) ) return null; else return f.getDoohickey(); }
I believe the "Elvis" operator was developed to solve this annoyance (return t.?getWhatsit().?getFoobar().?getDoohickey();) but it did not make the cut for Java 7 or even Java 8, from what I understand. Scala's solution to this issue is to recommend that operations which might not have a value for you return Option[ T ] instead of T. You can then map your method call to the Option and get back another Option without ever seeing a null pointer exception. Option is a variant type, easily modeled in Scala but not in Java (see 5).
Iterators. They can't throw checked exceptions. They have to implement remove(), often by throwing (unchecked) UnsupportedOperationExceptions. For-each syntax can't work with iterators directly. None of these problems arise with the superb collections framework in Scala which is designed hand-in-hand with its clean higher-order programming (see 4).
void. This is a holdover from C, and is obviously not anything Java can get rid of, but it's stupid. Because of void, e.g., I can never do a visitor pattern with just one kind of visitor; there has to be one whose methods return a generic type T, and another whose methods return void. And don't try to sell me on the psuedo-type Void, because you still have to accept or return null somewhere. Scala has a type Unit with a single trivial value (), and unitary methods/functions can explicitly return that value or not return anything; the semantics are the same. Thus all expressions have some meaningful type, and classes with generic types can be fully general.

Why tabs are better

2010-04-07T17:44:00.000-07:00

I'm tired of this stupid "tabs vs. spaces" code style debate. Tabs win hands down on just about every measure. Anyone still laboring under the misapprehension that it makes sense to indent one's source file using spaces should consider the following:

Line-based comments (‘//’, ‘#’) at the head of the line don’t screw up the indentation (unless tab depth <= 2).
You can change the indentation depth without editing the file. This is a huge feature, folks. If I like shallow indentation on all my source, I can make it so, and people who prefer the other extreme are not affected. The counter-argument (put forth by Checkstyle, among others) that one should not be required to set tab depth in order to read source is absurd; tab depth is always set to something, whether you like it or not (see 11), and code indented using tabs is readable regardless of the setting, unless tab depth is ridiculously high. The only code that actually does require a fixed tab depth to be legible is code that mixes tabs and spaces, which I encounter all too often. See 10.
Spaces-based indentation will inevitably become inconsistent because no one can agree on his/her favorite indentation depth (see 2).
Indentation mistakes are more obvious using tabs (unless tab depth = 1, which is just stupid).
Tab indentation characters, when used properly, are more semantically relevant than spaces.
Files are smaller (relevant especially for Javascript, CSS, HTML).
Fewer keystrokes are needed to navigate within source files. Sorry, but “Ctrl+Right arrow” is two keystrokes, plus you have to hold one of them down.
Making tabbed whitespace visible in an IDE is useful for eyeballing how things line up; making spaces visible is useful for “magic eye”.
Tabs are unable to support the unreadable, but nevertheless default, function-call line-break style of making parameters line up with the opening ‘(’. Remember, it is a feature that this abomination is not supportable. Unfortunately it is still possible to put just the first parameter on the same line as the ‘(’, but no indentation choice can prevent that bad decision.
If you have to edit a production config file using terminal-based default emacs, should you really be checking that in? I should add that the indentation used by default in Emacs (and pervasive in high-profile source such as the JDK) is a horrific hybrid of spaces and tabs which actually does force you to set your tab depth to a fixed value of 8 in order to read code thus indented. See 2.
Some well-known tools (e.g., ReviewBoard) typically display tabs with a depth of 8, which is kind of high. I claim that this tab discrimination is also a feature, because it discourages deeply-nested code which is a good thing.

The only moderately sane argument in favor of spaces is that the code "always looks the same". Isn't that nice. You can write comments that use little "^^^^" to point to something on the line above. Wow. I guess that's worth throwing out points 1-11.

I'm not going to wade into the quagmire of my other personal code style choices. But it's time this debate, which rages again and again every time I join a new team, be permanently put to bed.

JAXB, @XmlMixed, and white space anomalies

2009-06-25T13:44:00.000-07:00

Whether or not you think "mixed" content in XML is ever a good idea, you may need to handle it using JAXB one day. Recall that for JAXB to parse a mixed content XML element to a class C, you use an @XmlMixed annotation on a field of C of type List< Serializable >, combined with either @XmlAnyElement or @XmlElements. In each case, the resulting list will contain Strings representing the text nodes and objects representing the element nodes, in the same order as they appear in the XML text. Thus


<thing>stuff<nested/>entities<alsoNested/></thing>

maps to an instance of


@XmlRootElement
class Thing {
    @XmlMixed @XmlAnyElement
    List< Serializable > lserComponents;
}

which looks like


{ lserComponents : [ "stuff", { localName : "nested" }, "entities", { localName: "alsoNested" } ] }

Unfortunately, if the only content other than nested elements happens to be white space, as in


<thing><nested/>   <alsoNested/></thing>

you get the odd bound object


{ lserComponents : [ { localName : "nested" }, { localName: "alsoNested" }, "" ] }

If you care about white space, and who doesn't these days in the throes of late-stage Reaganomics, you need a trick when you actually go to parse the XML.

First, we create a SAX 2.0 ContentHandler implementation that delegates all events to a JAXB UnmarshallerHandler, but modifies all the whitespace slightly:


class WhitespaceAwareUnmarshallerHandler implements ContentHandler {
  private final UnmarshallerHandler uh;
  public WhitespaceAwareUnmarshallerHandler( UnmarshallerHandler uh ) {
    this.uh = uh;
  }
  /**
   * Replace all-whitespace character blocks with the character '\u000B',
   * which satisfies the following properties:
   * 
   * 1. "\u000B".matches( "\\s" ) == true
   * 2. when parsing XmlMixed content, JAXB does not suppress the whitespace
   **/
  public void characters(
    char[] ch, int start, int length
  ) throws SAXException {
    for ( int i = start + length - 1; i >= start; --i )
      if ( !Character.isWhitespace( ch[ i ] ) ) {
        uh.characters( ch, start, length );
        return;
      }
    Arrays.fill( ch, start, start + length, '\u000B' );
    uh.characters( ch, start, length );
  }
  /* what follows is just blind delegation monkey code */
  public void ignorableWhitespace( char[] ch, int start, int length ) throws SAXException { uh.characters( ch, start, length ); }
  public void endDocument() throws SAXException { uh.endDocument(); }
  public void endElement( String uri, String localName, String name ) throws SAXException { uh.endElement( uri,  localName, name ); }
  public void endPrefixMapping( String prefix ) throws SAXException { uh.endPrefixMapping( prefix ); }
  public void processingInstruction( String target, String data ) throws SAXException { uh.processingInstruction(  target, data ); }
  public void setDocumentLocator( Locator locator ) { uh.setDocumentLocator( locator ); }
  public void skippedEntity( String name ) throws SAXException { uh.skippedEntity( name ); }
  public void startDocument() throws SAXException { uh.startDocument(); }
  public void startElement( String uri, String localName, String name, Attributes atts ) throws SAXException { uh.startElement( uri, localName, name, atts ); }
  public void startPrefixMapping( String prefix, String uri ) throws SAXException { uh.startPrefixMapping( prefix, uri ); }
}

Then at parse time, instead of the usual ctx.createUnmarhaller().unmarshal( strData ), we substitute our special handler to do the parsing:


public class JAXBUtil {
  @SuppressWarnings( "unchecked" )
  public static < T > T unmarshal(
    JAXBContext ctx, String strData, boolean flgWhitespaceAware
  ) throws Exception {
    UnmarshallerHandler uh = ctx.createUnmarshaller().getUnmarshallerHandler();
    XMLReader xr = new WstxSAXParser(); // use your favorite SAX 2.0 parser
    xr.setContentHandler( flgWhitespaceAware ? new WhitespaceAwareUnmarshallerHandler( uh ) : uh );
    xr.parse( new InputSource( new StringReader( strData ) ) );
    return ( T )uh.getResult();
  }
}

Identity transformation, my butt

2008-05-15T17:16:00.000-07:00

Some lovely trivia I have recently discovered about the default implementations of XSLT transformations in the JDK 1.5:

The so-called "identity transformation" available at TransformerFactory.newTransformer() is anything but the identity when applied to XHTML, until certain non-default configuration is applied. Specifically, you have to do all this:
```
  xfmEng.setOutputProperty( OutputKeys.DOCTYPE_SYSTEM, "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd" );
  xfmEng.setOutputProperty( OutputKeys.DOCTYPE_PUBLIC, "-//W3C//DTD XHTML 1.0 Transitional//EN" );
  xfmEng.setOutputProperty( OutputKeys.METHOD, "html" );
  xfmEng.setOutputProperty( OutputKeys.OMIT_XML_DECLARATION, "yes" );
```
or you get tons of  garbage before the real document. The garbage seems to live in the w3c.org dtd files for xhtml.

Even with all that, you still end up with the very non-identity transformation of input like <script src=...></script> becoming <script src=.../>. The latter is actually malformed according to many browsers. Forget newTransformer() and use an xslt-based transformation like
```
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 <xsl:output method="html"/>
 <xsl:template match="/">
  <xsl:copy-of select="."/>
 </xsl:template>
</xsl:stylesheet>
```

Speaking of the w3c.org dtd files, there's still some nasty stuff going on behind the scenes; when any transformer created via TransformerFactory.newTransformer() or Templates.newTransformer() starts processing XHTML, it actually goes and grabs those extremely well-known DTDs off the web from their URIs at w3c.org. Every document, even with the same transformer, engenders a new set of GETs to w3c. Pretty ridiculous. Here's how to get around that:


package MyPackage;

import org.xml.sax.SAXNotRecognizedException;
import org.xml.sax.SAXNotSupportedException;
import com.sun.org.apache.xerces.internal.impl.Constants;
import com.sun.org.apache.xerces.internal.parsers.SAXParser;

public class MyTransform {

 // ...

 public static class MySAXParser extends SAXParser {
  public MySAXParser() {
   super();
   try {
    setFeature( Constants.SAX_FEATURE_PREFIX + Constants.VALIDATION_FEATURE, false );
    setFeature( Constants.XERCES_FEATURE_PREFIX + Constants.LOAD_EXTERNAL_DTD_FEATURE, false );
   } catch ( SAXNotRecognizedException sne ) {
   } catch ( SAXNotSupportedException sse ) {
   }
  }
 }

 // in the code that uses the transformer:
  System.setProperty( "org.xml.sax.driver", "MyPackage.MyTransform$MySAXParser" );
  TransformerFactory.newInstance().newTransformer().transform( stmIn, stmOut );

 // ...
}

AspectJ: A crucial distinction between execution and call join points

2008-05-15T17:14:00.000-07:00

Given the following code:


  class Super {
    protected void m() {}
  }
  class Sub extends Super {}

A call( * Sub+.m( .. ) ) join point will match any call to m made on an instance statically known to be a Sub (in particular, ( ( Super )s ).m() will not only not match, but will cause a compiler warning); an execution( * Sub+.m() ) join point will not match s.m(), even if s is Sub, because there's no m() code to execute in Sub!

Spring declarative transactions, applied dynamically

2008-05-15T13:01:00.000-07:00

I think declarative transaction management is possibly the most compelling reason for using the Spring Framework. I won't explain how it usually works, leaving that to the Spring docs and plenty of examples in the literature. Recently I ran into the following conundrum, however.

I have a service class that has not only some "transactional" methods, but also other methods that return instances of secondary classes with their own transactional methods. It is easy in the configuration of the Spring container to declare the transactional methods in the service as such, because it is a typical singleton bean. The problem is that if the code in the service implementation is to remain decoupled from Spring, how do we instruct Spring to wrap transactional proxies around any instances of the secondary classes that are produced at runtime?

My first attempt to solve this problem was to factor the code that needed to be transactional out of the secondary classes into package-access methods on the main service, in the hopes that I could declare those helper methods transactional in the configuration. Then the formerly transactional methods in the secondary classes could call those helper methods back in the service bean. This attempt proved to be not just inelegant, but also ineffectual: a complete waste of time.

Why didn't it work? Because the newly-minted secondary class instances were not receiving a reference to the service bean's transactional proxy to make their call backs on; they got the unadorned service instance as obtained via this, and in order to get the proxy, the Spring coupling would have to creep back into the code. Only a more complete AOP framework like AspectJ, with its special compiler, could avoid the this problem.

My solution to this problem was in four parts. Spring 2.5 is required.

Create a new implementation of each secondary class which is a pure delegating proxy. The constructor of DelegatingSecondaryClassImpl takes a SecondaryClass as delegate, and every method simply forwards to that delegate.

Create a BeanFactoryAware aspect ServiceAdvice with around advice for the methods on the service class that produce instances of these secondary classes, one advice method per secondary class. In each case the advice looks like this:
```
 public SecondaryClass secondaryClassAroundAdvice( ProceedingJoinPoint pjp ) throws Throwable {
  return ( SecondaryClass )bf.getBean( "secondaryClassBeanId", new Object[] { pjp.proceed() } );
 }
```

Add a prototype bean to the configuration XML for each secondary class:


 <bean id="secondaryClassBeanId" class="com.amazon.foo.bar.DelegatingSecondaryClassImpl" scope="prototype">
  <constructor-arg><null/></constructor-arg>
 </bean>

Add advice to the service methods that produce instances of each secondary class in the configuration XML:


 <bean id="serviceAdvice" class="com.amazon.foo.bar.ServiceAdvice">
 <aop:config>
  <!-- transactional advisor goes here ... -->
  <aop:aspect id="proxyAspect" ref="proxyAdvice">
   <aop:around pointcut="execution( com.amazon.foo.bar.SecondaryClass createSecondaryClass1( .. ) ) || execution( com.amazon.foo.bar.SecondaryClass createSecondaryClass2( .. ) )" method="secondaryClassAroundAdvice">
   </aop:around>
  </aop:aspect>
 </aop:config>

AspectJ for lazy loading, improved

2007-08-03T21:56:00.000-07:00

I find Russ Miles' lazy feature loading recipe (AspectJ Cookbook, chapter 21.3) cumbersome and at odds with the whole point of AOP. You have to use joinPoint.getArgs inside a proxy; but you're using AspectJ instead of Spring to avoid the proxy-based Spring AOP implementation, with all its intrinsic problems (what does "this" mean?). You also have to be aware in the mainline code that you're doing lazy loading and explicitly call LazyLoading.aspectOf().initializeFeature(), so it's intrusive.

In addition, there is essentially no reusability here. You have to create a special aspect for each interface which you wish to load lazily, with stubs for every method of that interface.

Finally, you have to use a weird trick of declaring that one interface implements another without actually giving any implementation (this line is given with no explanation in the text).

My approach, by contrast, is extremely simple. I do still impose a burden on the mainline code; specifically, if you want to see any benefit from lazy loading of an object you have to make sure the expensive initialization code actually occurs within the construction of that object. But that's it. No proxies, no hard-coded interface stubs, no intrusion.

The basic technique is to suppress the execution of the object's initialization code, wrapping it in a closure and storing it away until the first access to the object that requires that the initialization take place. The storage of the closure, and the state of the flag indicating that the initialization has or has not been attempted, live inside a small pertarget aspect.

Here's two variations of the LazyInitialization aspect.

The first:

// one aspect instance per lazily-instantiated object
public aspect LazyInitialization pertarget( target( Lazy ) ) {

// this interface represents a closure for the real initialization
private interface DelayedInit {
 void init();
}

// this is the marker interface for types that should be
// lazily initialized
public interface Lazy {};

 // per-object aspect state
 boolean flgTried = false;
 DelayedInit fnInit = null;

 // this advice intercepts the normal initialization process for
 // a Lazy object and stashes a closure for it in the aspect
 void around() : execution( Lazy+.new( .. ) ) {
  synchronized( this ) {
   if ( fnInit == null )
    fnInit = new DelayedInit() {
     public void init() {
      proceed();
     }
    };
  }
 }

  // this advice intercepts any access to a Lazy object that might
  // require it to actually initialize, and does so if necessary
  before() : call( * Lazy+.*( .. ) ) || get( * Lazy+.* ) || set( * Lazy+.* ) {
   synchronized( this ) {
    if ( !flgTried ) {
     flgTried = true;
     fnInit.init();
    }
   }
  }
}

To use it, just declare somewhere that some class or interface C implements LazyInitialization.Lazy and every instance of C or its subclasses/implementations will be lazily initialized:

aspect MakeCLazy {
   declare parents : C+ implements LazyInitialization.Lazy;
}

The other variation uses a generic type variable T instead of a marker interface. This version allows you to be more specific about which classes/interfaces/methods are to be lazily initialized and what constitutes an initialization-worthy access via filter pointcuts.

public abstract aspect GenericLazyInit< T > pertarget( initFilter() ) {

 // this interface represents a closure for the real initialization
 private interface DelayedInit {
  void init();
 }

 // per-object aspect state
 boolean flgTried = false;
 DelayedInit fnInit = null;

 protected pointcut initFilter() : execution( T+.new( .. ) );
 protected pointcut accessFilter() : call( * T+.*( .. ) ) || get( * T+.* ) || set( * T+.* );
 
 // this advice intercepts the normal initialization process for
 // a Lazy object and stashes a closure for it in the aspect
 void around() : initFilter() {
  synchronized( this ) {
   if ( fnInit == null )
    fnInit = new DelayedInit() {
     public void init() {
      proceed();
     }
    };
  }
 }

 // this advice intercepts any access to a Lazy object that might
 // require it to actually initialize, and does so if necessary
 before() : accessFilter() {
  synchronized( this ) {
   if ( !flgTried && fnInit != null ) {
    flgTried = true;
    fnInit.init();
   }
  }
 }
}

Using this version (in its default configuration, equivalent to variation 1) is even briefer:

aspect MakeCLazy extends GenericLazyInit< C > {}

Just in case you hadn't read about the Republicans who are running for congress in 2 weeks

2006-10-25T07:19:00.000-07:00

Here they are:

Jon Kyl Rick Renzi J.D. Hayworth John Doolittle Richard Pombo Brian Bilbray Marilyn Musgrave Doug Lamborn Rick O'Donnell Christopher Shays Vernon Buchanan Joe Negron Clay Shaw Bill Sali Peter Roskam Mark Kirk Dennis Hastert Chris Chocola John Hostettler Mike Whalen Jim Ryun Anne Northup Geoff Davis Michael Steele Gil Gutknecht Michele Bachmann Jim Talent Conrad Burns Jon Porter Charlie Bass Mike Ferguson Heather Wilson Peter King John Sweeney Tom Reynolds Randy Kuhl Robin Hayes Charles Taylor Steve Chabot Jean Schmidt Deborah Pryce Joy Padgett Melissa Hart Curt Weldon Mike Fitzpatrick Don Sherwood Lincoln Chafee Bob Corker George Allen Frank Wolf Mike McGavick Dave Reichert

But...but...we haven't been attacked again since 9/11!

2006-10-10T22:24:00.000-07:00

May I remind you that we've now lost about as many soldiers in Iraq as we lost civilians. Iraqis have lost something like 8 to 20 times as many civilians. Terrorist attacks have skyrocketed since 9/11 worldwide. We weren't attacked for five years before 9/11 either on U.S. soil, unless you count the Oklahoma City bombing.

What, you want a smug response to this canard? OK, how about: that's like saying we didn't get robbed again since we started leaving stacks of $100 bills on the corner every night.

Any other good metaphors?

I like strong polymorphic type disciplines. I dislike Republicans.

2006-03-04T16:54:00.001-08:00

The title says it all. Convince me otherwise.