Friday, August 05, 2011

Requirements for using a Hadoop combiner

One way to control I/O for large Hadoop jobs, especially those whose mappers produce many records with relatively fewer distinct keys, is to introduce a combiner phase between the mapper and reducer. I have not seen a simple diagram explaining what the types should be for the combiner and what properties the algorithm must exhibit, so here goes. If your mapper extends Mapper< K1, V1, K2, V2 > and your reducer extends Reducer< K2, V2, K3, V3 >, then the combiner must be an extension of Reducer< K2, V2, K2, V2 >, and the following diagram must commute:

The triangle in the center of the diagram represents distributivity of the combiner function (i.e., reduce(k, combine(k, M)) = reduce(k, combine(k, ∪icombine(k,σi(M))) for any partition σ = {σi | i ∈ I} of M), because Hadoop does not guarantee how many times it will combine intermediate outputs, if at all.

A common pattern is to use the same function for combiner and reducer, but for this pattern to work, we must have K2 = K3 and V2 = V3 (and of course, the reducer itself must be distributive).

I should also mention that if you use a grouping comparator for your Reducer that is different from your sorting comparator, the above diagram is not correct. Unfortunately, and I'm pretty sure this is an outright bug in Hadoop, the sorting comparator is always used to group inputs for the Combiner's reduce() calls (see MAPREDUCE-3310).

Thursday, June 16, 2011

Escaping property placeholders in Spring XML config

The problem

You might have encountered the awkward situation in which you are
  1. using Spring and XML config
  2. substituting properties into that config via some PropertyPlaceholderConfigurator
  3. needing to set some value in the config to a literal string of the form "${identifier}"

By default, any string of the form in 3. above is a placeholder, and if you have no value for that placeholder, you get an exception. Spring JIRA issue SPR-4953, which recognizes the fact that there is no simple escaping syntax for placeholders, is still open as of this writing.

A snippet such as the following will cause the exception if there is no value available to substitute for the variable customerName, or actually substitute a value for it if it is available. Neither result is desirable in our scenario; we want the "${customerName}" to remain intact when it is injected into our bean.
<bean id="aTrickyBean" class="org.anic.veggies.AreGoodForYou">
    <constructor-arg name="expression" value="Hello, ${customerName}!"/>
</bean>

Most workarounds I have seen are unsatisfactory. You can use a customized placeholder configurator that sets its delimiter characters to something other than the default, for example, which would mean you would have to change the look of all the actual (unescaped) placeholders just to support the ones you want escaped.

The workaround

However, in Spring 3.x, you can work around this issue in a much more simple way using the following trick with SpEL:
<bean id="aTrickyBean" class="org.anic.veggies.AreGoodForYou">
    <constructor-arg name="expression" value="#{ 'Hello, $' + '{customerName}!' }"/>
</bean>
Note that in order for this trick to work it is vital that the '$' and the '{' be physically separated (in this case, on either side of a string concatenation).

Thursday, November 11, 2010

Random notes on Hadoop

I am talking about Hadoop 0.20 using a custom jar, not streaming or Hive or Pig.
  1. Make sure your distribution has the MAPREDUCE-1182 patch.
  2. Make sure you change the default setting of dfs.datanode.max.xcievers to something very large, like 4096. And yes, the property name is misspelled. In 0.22/0.23 the property will be called dfs.datanode.max.transfer.threads.
  3. If you've declared your key type to be T, you can't write an S, even if S is a subclass of T.

  4. There are several ways to get your dependencies visible to Hadoop tasks and tools, and they are all clunky.
    • You can bundle them all into the job jar in a folder called lib, although doing so does not make it possible to have custom input formats, output formats, mappers, and reducers in separate jars.
    • You can use the -libjars argument, but if you ever have to load a class via reflection (i.e., using Class.forName), you have to use the Thread.currentThread().getContextClassLoader() rather than the default. Also you might run into HADOOP-6103.
    • You can use DistributedCache.addFileToClassPath, but you have to be sure to put the file on HDFS and refer to it by its absolute pathname without a scheme or authority, and these files are only available to the tasks, not the tool/job.

  5. DistributedCache is just plain wonky. You must
    1. put your file on hdfs somehow
    2. call DistributedCache.addCacheFile(), using the full URI with "hdfs:///"
    3. in the mapper/reducer, use java.io.* APIs to access the files represented by the paths in DistributedCache.getLocalCacheFiles(). Incredibly, the "typical usage" example in the javadocs for DistributedCache just completely elides this crucial bit. If you try to use FileSystem.get().open(), you'll get a cryptic error message with a filename that looks like it's been mangled.
      I can't find a programmatic mapping between files added via addCacheFile() and paths retrieved by getLocalCacheFiles(), although there may be some support for sharing the name or specifying it with the "hdfs://source#target" syntax. None of this API is well-documented.
  6. You can't substitute Hadoop counters for actual aggregation in your reducer, tempting as that might be. Counters will differ from run to run, even against identical inputs, because of things that vary like speculative execution and task failures.
  7. If you configure your cluster to have 0 reduce slots (perhaps because your jobs are all map-only), and you accidentally submit a job that does require a reduce phase, that job will run all mappers to completion and then hang forever.

Sunday, October 24, 2010

6 10 Things I Hate About Java (or, Scala is the Way and the Light)

I've been working with Java quite extensively for about 4 years now, and it has been enjoyable for the most part. Garbage collection, the JVM, generics, anonymous classes, and superb IDE support have made my life much easier.

But a few things make me gnash my teeth on a daily basis, and it's funny, but none of them are issues in another JVM language in which I have been dabbling, Scala. It seems the developers of that language felt my pain as well.

  1. Miserable type inference. Apparently some of the problems with it are being addressed in project coin for Java 7. The blue portions of the following code are, to any sane programmer, maddeningly superfluous, but nevertheless strictly required in Java until at least mid-2011:

    List< Integer > li1 = new ArrayList< Integer >();
    List< Integer > li2 = Arrays.asList( 1, 2, 3 );
    o.processListOfInteger( Arrays.< Integer >asList() );

    Needless to say, equivalent initializations in Scala require no such redundant information.
  2. Generic invariance. An example from last week: I'm working on an implementation FrazzleExecutorService of java.util.concurrent.ScheduledExecutorService and a refinement FrazzleFuture< T > of java.util.ScheduledFuture< T >. Covariant subtyping lets me get away with returning a FrazzleFuture< T > from a method like FrazzleExecutorService.submit() without violating the contract. But I can't return List< FrazzleFuture< T > > from FrazzleExecutorService.invokeAll() because (a) ScheduledExecutorService would have had to declare the return type to be List< ? extends ScheduledFuture< T > >; (b) the returned list should have been immutable anyway; (c) generic types like List< T > are invariant in their parameters. In Scala, there is a separate mutable and immutable collections hierarchy, and at least in the immutable one, S <: T implies List[ S ] <: List[ T ], because List is declared covariant in its parameter.
  3. Collections can't be tersely initialized. Part of the blame is the Collections framework; part of it is the goddamn language. The following code illustrates, with green text indicating typical verbosity:

    List< Integer > li1 = new ArrayList< Integer >(
        Arrays.asList(
    1, 2, 3 )
    )
    ;
    @SuppressWarnings( "serial" )
    Map< String, String > mss1 = new HashMap< String, String >() { {
        put
    ( "foo", "bar" );
        put
    ( "this", "sucks" );
    } }
    ;

    Turns out the collections literals are also not supported in Scala; you have to type List( 1, 2, 3 ) or Map( "foo" -> "bar", "this" -> "rocks" ). Excuse me if I mock Java incessantly at this point. Collection improvements have been postponed until Java 8, scheduled for mid-2012.
  4. Higher-order programming is absurdly verbose. Rather than give code samples, I'll just refer you to these guys and let you see for yourself how even a library can't save you from massive boilerplate for the simplest things. And Scala? Lambda expressions are built-in as syntactic sugar for functional objects, making higher-order code simple, readable, and terse.
  5. Modeling variant types is awkward. You have to choose from among many bad options:
    RepresentationInterrogation
    one sparsely-populated class (S + T modeled as S × T)if-ladders based on comparing to null (see 8)
    S × T and an enum of type labelsswitch + casting
    a hierarchya bunch of isS() and isT() methods and casting
    a hierarchyvarious casting attempts wrapped with ClassCastException catch blocks (ok, that's not really an option, but I get that as an answer in interviews sometimes)
    a hierarchyif-ladders based on instanceof and casting
    a hierarchypolymorphic decomposition and the inevitable bloated APIs at the base class that result
    a hierarchy that includes Visitorpainfully verbose visitors (see 4 and 10)
    a hierarchy of Throwablesthrow and various catch blocks, which I suspect compiles to the same thing as the instanceof approach, only more expensive (but actually requires the least code!)
    Scala has case classes and pattern matching built in.
  6. No tuples. One ends up either creating Pair< S, T > or dozens of throwaway classes with "And" in the name, like CountAndElapsed. Scala has tuples, although I feel like they kind of screwed up by not going the full ML and making multi-argument functions/methods be the same as single-argument functions/methods over tuples. So to call a 2-arg function f with a pair p = ( p1, p2 ), you can either call f( p1, p2 ) or f.tupled( p ). There must be some deep reason for making the distinction.
  7. No mixins. If you need stuff from 2 abstract classes, you will be copying, or aggregating (with loads of monkey delegation boilerplate) at least one of the two.
  8. Null. The following code should illustrate:

    private static Doohickey getDoohickey( Thingamajigger t ) {
        Whatsit w;
        Foobar f;
        if ( null == t )
            return null;
        else if ( null == ( w = t.getWhatsit() ) )
            return null;
        else if ( null == ( f = w.getFoobar() ) )
            return null;
        else
            return f.getDoohickey();
    }

    I believe the "Elvis" operator was developed to solve this annoyance (return t.?getWhatsit().?getFoobar().?getDoohickey();) but it did not make the cut for Java 7 or even Java 8, from what I understand. Scala's solution to this issue is to recommend that operations which might not have a value for you return Option[ T ] instead of T. You can then map your method call to the Option and get back another Option without ever seeing a null pointer exception. Option is a variant type, easily modeled in Scala but not in Java (see 5).
  9. Iterators. They can't throw checked exceptions. They have to implement remove(), often by throwing (unchecked) UnsupportedOperationExceptions. For-each syntax can't work with iterators directly. None of these problems arise with the superb collections framework in Scala which is designed hand-in-hand with its clean higher-order programming (see 4).
  10. Void. This is a holdover from C, and is obviously not anything Java can get rid of, but it's stupid. Because of void, e.g., I can never do a visitor pattern with just one kind of visitor; there has to be one whose methods return a generic type T, and another whose methods return void. And don't try to sell me on the psuedo-type Void, because you still have to accept or return null somewhere. Scala has a type Unit with a single trivial value (), and unitary methods/functions can explicitly return that value or not return anything; the semantics are the same. Thus all expressions have some meaningful type, and classes with generic types can be fully general.

Wednesday, April 07, 2010

Why tabs are better

I'm tired of this stupid "tabs vs. spaces" code style debate. Tabs win hands down on just about every measure. Anyone still laboring under the misapprehension that it makes sense to indent one's source file using spaces should consider the following:
  1. Line-based comments (‘//’, ‘#’) at the head of the line don’t screw up the indentation (unless tab depth <= 2).
  2. You can change the indentation depth without editing the file. This is a huge feature, folks. If I like shallow indentation on all my source, I can make it so, and people who prefer the other extreme are not affected. The counter-argument (put forth by Checkstyle, among others) that one should not be required to set tab depth in order to read source is absurd; tab depth is always set to something, whether you like it or not (see 11), and code indented using tabs is readable regardless of the setting, unless tab depth is ridiculously high. The only code that actually does require a fixed tab depth to be legible is code that mixes tabs and spaces, which I encounter all too often. See 10.
  3. Spaces-based indentation will inevitably become inconsistent because no one can agree on his/her favorite indentation depth (see 2).
  4. Indentation mistakes are more obvious using tabs (unless tab depth = 1, which is just stupid).
  5. Tab indentation characters, when used properly, are more semantically relevant than spaces.
  6. Files are smaller (relevant especially for Javascript, CSS, HTML).
  7. Fewer keystrokes are needed to navigate within source files. Sorry, but “Ctrl+Right arrow” is two keystrokes, plus you have to hold one of them down.
  8. Making tabbed whitespace visible in an IDE is useful for eyeballing how things line up; making spaces visible is useful for “magic eye”.
  9. Tabs are unable to support the unreadable, but nevertheless default, function-call line-break style of making parameters line up with the opening ‘(’. Remember, it is a feature that this abomination is not supportable. Unfortunately it is still possible to put just the first parameter on the same line as the ‘(’, but no indentation choice can prevent that bad decision.
  10. If you have to edit a production config file using terminal-based default emacs, should you really be checking that in? I should add that the indentation used by default in Emacs (and pervasive in high-profile source such as the JDK) is a horrific hybrid of spaces and tabs which actually does force you to set your tab depth to a fixed value of 8 in order to read code thus indented. See 2.
  11. Some well-known tools (e.g., ReviewBoard) typically display tabs with a depth of 8, which is kind of high. I claim that this tab discrimination is also a feature, because it discourages deeply-nested code which is a good thing.

The only moderately sane argument in favor of spaces is that the code "always looks the same". Isn't that nice. You can write comments that use little "^^^^" to point to something on the line above. Wow. I guess that's worth throwing out points 1-11.

I'm not going to wade into the quagmire of my other personal code style choices. But it's time this debate, which rages again and again every time I join a new team, be permanently put to bed.

Thursday, June 25, 2009

JAXB, @XmlMixed, and white space anomalies

Whether or not you think "mixed" content in XML is ever a good idea, you may need to handle it using JAXB one day. Recall that for JAXB to parse a mixed content XML element to a class C, you use an @XmlMixed annotation on a field of C of type List< Serializable >, combined with either @XmlAnyElement or @XmlElements. In each case, the resulting list will contain Strings representing the text nodes and objects representing the element nodes, in the same order as they appear in the XML text. Thus

<thing>stuff<nested/>entities<alsoNested/></thing>
maps to an instance of
@XmlRootElement
class Thing {
    @XmlMixed @XmlAnyElement
    List< Serializable > lserComponents;
}
which looks like
{ lserComponents : [ "stuff", { localName : "nested" }, "entities", { localName: "alsoNested" } ] }
Unfortunately, if the only content other than nested elements happens to be white space, as in
<thing><nested/>   <alsoNested/></thing>
you get the odd bound object
{ lserComponents : [ { localName : "nested" }, { localName: "alsoNested" }, "" ] }
If you care about white space, and who doesn't these days in the throes of late-stage Reaganomics, you need a trick when you actually go to parse the XML.

First, we create a SAX 2.0 ContentHandler implementation that delegates all events to a JAXB UnmarshallerHandler, but modifies all the whitespace slightly:

class WhitespaceAwareUnmarshallerHandler implements ContentHandler {
  private final UnmarshallerHandler uh;
  public WhitespaceAwareUnmarshallerHandler( UnmarshallerHandler uh ) {
    this.uh = uh;
  }
  /**
   * Replace all-whitespace character blocks with the character '\u000B',
   * which satisfies the following properties:
   * 
   * 1. "\u000B".matches( "\\s" ) == true
   * 2. when parsing XmlMixed content, JAXB does not suppress the whitespace
   **/
  public void characters(
    char[] ch, int start, int length
  ) throws SAXException {
    for ( int i = start + length - 1; i >= start; --i )
      if ( !Character.isWhitespace( ch[ i ] ) ) {
        uh.characters( ch, start, length );
        return;
      }
    Arrays.fill( ch, start, start + length, '\u000B' );
    uh.characters( ch, start, length );
  }
  /* what follows is just blind delegation monkey code */
  public void ignorableWhitespace( char[] ch, int start, int length ) throws SAXException { uh.characters( ch, start, length ); }
  public void endDocument() throws SAXException { uh.endDocument(); }
  public void endElement( String uri, String localName, String name ) throws SAXException { uh.endElement( uri,  localName, name ); }
  public void endPrefixMapping( String prefix ) throws SAXException { uh.endPrefixMapping( prefix ); }
  public void processingInstruction( String target, String data ) throws SAXException { uh.processingInstruction(  target, data ); }
  public void setDocumentLocator( Locator locator ) { uh.setDocumentLocator( locator ); }
  public void skippedEntity( String name ) throws SAXException { uh.skippedEntity( name ); }
  public void startDocument() throws SAXException { uh.startDocument(); }
  public void startElement( String uri, String localName, String name, Attributes atts ) throws SAXException { uh.startElement( uri, localName, name, atts ); }
  public void startPrefixMapping( String prefix, String uri ) throws SAXException { uh.startPrefixMapping( prefix, uri ); }
}
Then at parse time, instead of the usual ctx.createUnmarhaller().unmarshal( strData ), we substitute our special handler to do the parsing:
public class JAXBUtil {
  @SuppressWarnings( "unchecked" )
  public static < T > T unmarshal(
    JAXBContext ctx, String strData, boolean flgWhitespaceAware
  ) throws Exception {
    UnmarshallerHandler uh = ctx.createUnmarshaller().getUnmarshallerHandler();
    XMLReader xr = new WstxSAXParser(); // use your favorite SAX 2.0 parser
    xr.setContentHandler( flgWhitespaceAware ? new WhitespaceAwareUnmarshallerHandler( uh ) : uh );
    xr.parse( new InputSource( new StringReader( strData ) ) );
    return ( T )uh.getResult();
  }
}

Thursday, May 15, 2008

Identity transformation, my butt

Some lovely trivia I have recently discovered about the default implementations of XSLT transformations in the JDK 1.5:


  1. The so-called "identity transformation" available at TransformerFactory.newTransformer() is anything but the identity when applied to XHTML, until certain non-default configuration is applied. Specifically, you have to do all this:


    xfmEng.setOutputProperty( OutputKeys.DOCTYPE_SYSTEM, "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd" );
    xfmEng.setOutputProperty( OutputKeys.DOCTYPE_PUBLIC, "-//W3C//DTD XHTML 1.0 Transitional//EN" );
    xfmEng.setOutputProperty( OutputKeys.METHOD, "html" );
    xfmEng.setOutputProperty( OutputKeys.OMIT_XML_DECLARATION, "yes" );


    or you get tons of <!-- ... --> garbage before the real document. The garbage seems to live in the w3c.org dtd files for xhtml.

  2. Even with all that, you still end up with the very non-identity transformation of input like <script src=...></script> becoming <script src=.../>. The latter is actually malformed according to many browsers. Forget newTransformer() and use an xslt-based transformation like

    <?xml version="1.0" encoding="UTF-8"?>
    <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="html"/>
    <xsl:template match="/">
    <xsl:copy-of select="."/>
    </xsl:template>
    </xsl:stylesheet>

  3. Speaking of the w3c.org dtd files, there's still some nasty stuff going on behind the scenes; when any transformer created via TransformerFactory.newTransformer() or Templates.newTransformer() starts processing XHTML, it actually goes and grabs those extremely well-known DTDs off the web from their URIs at w3c.org. Every document, even with the same transformer, engenders a new set of GETs to w3c. Pretty ridiculous. Here's how to get around that:


    package MyPackage;

    import org.xml.sax.SAXNotRecognizedException;
    import org.xml.sax.SAXNotSupportedException;
    import com.sun.org.apache.xerces.internal.impl.Constants;
    import com.sun.org.apache.xerces.internal.parsers.SAXParser;

    public class MyTransform {

    // ...

    public static class MySAXParser extends SAXParser {
    public MySAXParser() {
    super();
    try {
    setFeature( Constants.SAX_FEATURE_PREFIX + Constants.VALIDATION_FEATURE, false );
    setFeature( Constants.XERCES_FEATURE_PREFIX + Constants.LOAD_EXTERNAL_DTD_FEATURE, false );
    } catch ( SAXNotRecognizedException sne ) {
    } catch ( SAXNotSupportedException sse ) {
    }
    }
    }

    // in the code that uses the transformer:
    System.setProperty( "org.xml.sax.driver", "MyPackage.MyTransform$MySAXParser" );
    TransformerFactory.newInstance().newTransformer().transform( stmIn, stmOut );

    // ...
    }


AspectJ: A crucial distinction between execution and call join points

Given the following code:


class Super {
protected void m() {}
}
class Sub extends Super {}


A call( * Sub+.m( .. ) ) join point will match any call to m made on an instance statically known to be a Sub (in particular, ( ( Super )s ).m() will not only not match, but will cause a compiler warning); an execution( * Sub+.m() ) join point will not match s.m(), even if s is Sub, because there's no m() code to execute in Sub!

Spring declarative transactions, applied dynamically

I think declarative transaction management is possibly the most compelling reason for using the Spring Framework. I won't explain how it usually works, leaving that to the Spring docs and plenty of examples in the literature. Recently I ran into the following conundrum, however.

I have a service class that has not only some "transactional" methods, but also other methods that return instances of secondary classes with their own transactional methods. It is easy in the configuration of the Spring container to declare the transactional methods in the service as such, because it is a typical singleton bean. The problem is that if the code in the service implementation is to remain decoupled from Spring, how do we instruct Spring to wrap transactional proxies around any instances of the secondary classes that are produced at runtime?

My first attempt to solve this problem was to factor the code that needed to be transactional out of the secondary classes into package-access methods on the main service, in the hopes that I could declare those helper methods transactional in the configuration. Then the formerly transactional methods in the secondary classes could call those helper methods back in the service bean. This attempt proved to be not just inelegant, but also ineffectual: a complete waste of time.

Why didn't it work? Because the newly-minted secondary class instances were not receiving a reference to the service bean's transactional proxy to make their call backs on; they got the unadorned service instance as obtained via this, and in order to get the proxy, the Spring coupling would have to creep back into the code. Only a more complete AOP framework like AspectJ, with its special compiler, could avoid the this problem.

My solution to this problem was in four parts. Spring 2.5 is required.


  1. Create a new implementation of each secondary class which is a pure delegating proxy. The constructor of DelegatingSecondaryClassImpl takes a SecondaryClass as delegate, and every method simply forwards to that delegate.

  2. Create a BeanFactoryAware aspect ServiceAdvice with around advice for the methods on the service class that produce instances of these secondary classes, one advice method per secondary class. In each case the advice looks like this:

    public SecondaryClass secondaryClassAroundAdvice( ProceedingJoinPoint pjp ) throws Throwable {
    return ( SecondaryClass )bf.getBean( "secondaryClassBeanId", new Object[] { pjp.proceed() } );
    }


  3. Add a prototype bean to the configuration XML for each secondary class:

    <bean id="secondaryClassBeanId" class="com.amazon.foo.bar.DelegatingSecondaryClassImpl" scope="prototype">
    <constructor-arg><null/></constructor-arg>
    </bean>


  4. Add advice to the service methods that produce instances of each secondary class in the configuration XML:

    <bean id="serviceAdvice" class="com.amazon.foo.bar.ServiceAdvice">
    <aop:config>
    <!-- transactional advisor goes here ... -->
    <aop:aspect id="proxyAspect" ref="proxyAdvice">
    <aop:around pointcut="execution( com.amazon.foo.bar.SecondaryClass createSecondaryClass1( .. ) ) || execution( com.amazon.foo.bar.SecondaryClass createSecondaryClass2( .. ) )" method="secondaryClassAroundAdvice">
    </aop:around>
    </aop:aspect>
    </aop:config>


Friday, August 03, 2007

AspectJ for lazy loading, improved

I find Russ Miles' lazy feature loading recipe (AspectJ Cookbook, chapter 21.3) cumbersome and at odds with the whole point of AOP. You have to use joinPoint.getArgs inside a proxy; but you're using AspectJ instead of Spring to avoid the proxy-based Spring AOP implementation, with all its intrinsic problems (what does "this" mean?). You also have to be aware in the mainline code that you're doing lazy loading and explicitly call LazyLoading.aspectOf().initializeFeature(), so it's intrusive.

In addition, there is essentially no reusability here. You have to create a special aspect for each interface which you wish to load lazily, with stubs for every method of that interface.

Finally, you have to use a weird trick of declaring that one interface implements another without actually giving any implementation (this line is given with no explanation in the text).

My approach, by contrast, is extremely simple. I do still impose a burden on the mainline code; specifically, if you want to see any benefit from lazy loading of an object you have to make sure the expensive initialization code actually occurs within the construction of that object. But that's it. No proxies, no hard-coded interface stubs, no intrusion.

The basic technique is to suppress the execution of the object's initialization code, wrapping it in a closure and storing it away until the first access to the object that requires that the initialization take place. The storage of the closure, and the state of the flag indicating that the initialization has or has not been attempted, live inside a small pertarget aspect.

Here's two variations of the LazyInitialization aspect.

The first:

// one aspect instance per lazily-instantiated object
public aspect LazyInitialization pertarget( target( Lazy ) ) {

// this interface represents a closure for the real initialization
private interface DelayedInit {
 void init();
}

// this is the marker interface for types that should be
// lazily initialized
public interface Lazy {};

 // per-object aspect state
 boolean flgTried = false;
 DelayedInit fnInit = null;

 // this advice intercepts the normal initialization process for
 // a Lazy object and stashes a closure for it in the aspect
 void around() : execution( Lazy+.new( .. ) ) {
  synchronized( this ) {
   if ( fnInit == null )
    fnInit = new DelayedInit() {
     public void init() {
      proceed();
     }
    };
  }
 }

  // this advice intercepts any access to a Lazy object that might
  // require it to actually initialize, and does so if necessary
  before() : call( * Lazy+.*( .. ) ) || get( * Lazy+.* ) || set( * Lazy+.* ) {
   synchronized( this ) {
    if ( !flgTried ) {
     flgTried = true;
     fnInit.init();
    }
   }
  }
}

To use it, just declare somewhere that some class or interface C implements LazyInitialization.Lazy and every instance of C or its subclasses/implementations will be lazily initialized:

 
aspect MakeCLazy {
   declare parents : C+ implements LazyInitialization.Lazy;
}

The other variation uses a generic type variable T instead of a marker interface. This version allows you to be more specific about which classes/interfaces/methods are to be lazily initialized and what constitutes an initialization-worthy access via filter pointcuts.

public abstract aspect GenericLazyInit< T > pertarget( initFilter() ) {

 // this interface represents a closure for the real initialization
 private interface DelayedInit {
  void init();
 }

 // per-object aspect state
 boolean flgTried = false;
 DelayedInit fnInit = null;

 protected pointcut initFilter() : execution( T+.new( .. ) );
 protected pointcut accessFilter() : call( * T+.*( .. ) ) || get( * T+.* ) || set( * T+.* );
 
 // this advice intercepts the normal initialization process for
 // a Lazy object and stashes a closure for it in the aspect
 void around() : initFilter() {
  synchronized( this ) {
   if ( fnInit == null )
    fnInit = new DelayedInit() {
     public void init() {
      proceed();
     }
    };
  }
 }

 // this advice intercepts any access to a Lazy object that might
 // require it to actually initialize, and does so if necessary
 before() : accessFilter() {
  synchronized( this ) {
   if ( !flgTried && fnInit != null ) {
    flgTried = true;
    fnInit.init();
   }
  }
 }
}

Using this version (in its default configuration, equivalent to variation 1) is even briefer:

 
aspect MakeCLazy extends GenericLazyInit< C > {}

Tuesday, October 10, 2006

But...but...we haven't been attacked again since 9/11!

May I remind you that we've now lost about as many soldiers in Iraq as we lost civilians. Iraqis have lost something like 8 to 20 times as many civilians. Terrorist attacks have skyrocketed since 9/11 worldwide. We weren't attacked for five years before 9/11 either on U.S. soil, unless you count the Oklahoma City bombing.

What, you want a smug response to this canard? OK, how about: that's like saying we didn't get robbed again since we started leaving stacks of $100 bills on the corner every night.

Any other good metaphors?