Add data pack file format to the grit output type and enable it for

Add data pack file format to the grit output type and enable it for webkit_resources.grd.

This only adds support for <include> nodes.  <message> nodes are forthcoming.

The design doc describing the format is here:
http://dev.chromium.org/developers/design-documents/linuxresourcesandlocalizedstrings

I hooked into the <release> tag because we need to generate an index at the beginning of the file.  The index contains the position of all the resources, so we walk the file at that point.

This currently only handles <include> tags, but will be extended to handle <message> as well.

[tony, 2009-01-14 02:24:07]

Joi, this is the first cut at having GRIT generate a generic data file for use
on linux and maybe mac.  Can you take a look and provide some feedback?

Thanks!

[tony, 2009-01-15 22:38:02]

Joi emailed me comments, I'm including them here for record:
Suggest indicating in the change description that this adds support only for    
<include> nodes and that string tables or other formats are not yet             
supported.                                                                      
                                                                                
Strongly suggest adding a unit test for data_pack.py                            
                                                                                
include.py                                                                      
line 56: you should get the file handle to a temporary variable and             
explicitly close it after you're done with it                                   
                                                                                
data_pack.py                                                                    
                                                                                
Two blank lines between top-level elements per Google style guide (e.g.         
between functions, a function and a class, and around the kPackFileVersion      
constant)                                                                       
                                                                                
Suggest making GetDataNotes and WriteDataPack into static methods of the        
DataPack class (example follows):                                               
                                                                                
class DataPack(interface.ItemFormatter):                                        
  ...                                                                           
  @staticmethod                                                                 
  def WriteDataPack(resources):                                                 
    ...                                                                         
                                                                                
line 16: kPackFileConstant -> PACK_FILE_CONSTANT (per style guide)              
                                                                                
line 25: I doubt there's any need to change this, but for future reference      
note it would be more efficient to pass a list down via the recursive           
function and have it append to that list than to create a new empty list,       
build it, and then concatenate it to the other lists                            
                                                                                
A separate performance note is that it would be more efficient in terms of      
memory use to walk the tree once to get all the IDs and the size of the data    
items, then again to get the individual data items, keeping only one in         
memory at a time, only until it is written to the pack file.  The current       
approach is probably fine, but if you start seeing longer 'grit build'          
times, you might want to consider this.                                         
                                                                                
line 48: Just yet another performance note; it is much less expensive to        
build 'ret' up as a list of strings and then [ return ''.join(ret) ] at the     
very end rather than concatenating again and again to the same string.          
                                                                                
line 36: This is a constant so strictly speaking should be named                
INDEX_LENGTH.  Or you could just use [ len(ret) ].                              
                                                                                
Cheers,                                                                         
J�i

[tony, 2009-01-15 23:01:52]

I addressed all the comments except passing in a list to avoid list resizes.  If
it turns out to be slow, we can optimize.

Take another look?

On 2009/01/15 22:38:02, tony wrote:
> Joi emailed me comments, I'm including them here for record:
> Suggest indicating in the change description that this adds support only for  
 
> <include> nodes and that string tables or other formats are not yet           
 
> supported.                                                                    
 
>                                                                               
 
> Strongly suggest adding a unit test for data_pack.py                          
 
>                                                                               
 
> include.py                                                                    
 
> line 56: you should get the file handle to a temporary variable and           
 
> explicitly close it after you're done with it                                 
 
>                                                                               
 
> data_pack.py                                                                  
 
>                                                                               
 
> Two blank lines between top-level elements per Google style guide (e.g.       
 
> between functions, a function and a class, and around the kPackFileVersion    
 
> constant)                                                                     
 
>                                                                               
 
> Suggest making GetDataNotes and WriteDataPack into static methods of the      
 
> DataPack class (example follows):                                             
 
>                                                                               
 
> class DataPack(interface.ItemFormatter):                                      
 
>   ...                                                                         
 
>   @staticmethod                                                               
 
>   def WriteDataPack(resources):                                               
 
>     ...                                                                       
 
>                                                                               
 
> line 16: kPackFileConstant -> PACK_FILE_CONSTANT (per style guide)            
 
>                                                                               
 
> line 25: I doubt there's any need to change this, but for future reference    
 
> note it would be more efficient to pass a list down via the recursive         
 
> function and have it append to that list than to create a new empty list,     
 
> build it, and then concatenate it to the other lists                          
 
>                                                                               
 
> A separate performance note is that it would be more efficient in terms of    
 
> memory use to walk the tree once to get all the IDs and the size of the data  
 
> items, then again to get the individual data items, keeping only one in       
 
> memory at a time, only until it is written to the pack file.  The current     
 
> approach is probably fine, but if you start seeing longer 'grit build'        
 
> times, you might want to consider this.                                       
 
>                                                                               
 
> line 48: Just yet another performance note; it is much less expensive to      
 
> build 'ret' up as a list of strings and then [ return ''.join(ret) ] at the   
 
> very end rather than concatenating again and again to the same string.        
 
>                                                                               
 
> line 36: This is a constant so strictly speaking should be named              
 
> INDEX_LENGTH.  Or you could just use [ len(ret) ].                            
 
>                                                                               
 
> Cheers,                                                                       
 
> J�i