Cluster Analysis

Needs["Statistics`MultiDescriptiveStatistics`"]

This package is needed for the data: in general, the principal components of the original data are used in this  clustering procedure rather than the data itself.

The main function is recursive, so special cases are included to avoid infinite recursion

clusterData::usage = 
      "clusterData clusters data comprising points in multidimensional \
space into clusters of radius no greater than threshold."

\(clusterData[currentclusters : {__?MatrixQ}, data_?MatrixQ, 
          threshold_Real] /; 
        \((\((Length[data] > 1)\)\  && \ 
            \((And\ @@\ 
                \((\ Thread[
                    \((\(\(Dimensions[#]\)[[2
                              ]]\ &\)\ /@\ currentclusters)
                        \) == \(Dimensions[data]\)[[2
                        ]]\ ])\)\ )\)\ )\) := 
	
      clusterData[updateclusters[currentclusters, First[data], threshold], 
        Rest[data], threshold]\)

\(clusterData[currentclusters : {__?MatrixQ}, data : {_?VectorQ}, 
          threshold_Real]\  /; 
        \((And\ @@\ 
            \((\ Thread[
                \((\(\(Dimensions[#]\)[[2
                          ]]\ &\)\ /@\ currentclusters)
                    \) == \(Dimensions[data]\)[[2
                    ]]\ ])\)\ )\)\  := 
	
      updateclusters[currentclusters, First[data], threshold]\)

clusterData[currentclusters : {__?MatrixQ}, {{}}, threshold_Real] /; 
        (Equal @@
            ( (
                    Dimensions[#][[2
                      ]]) & /@ clusters)) :=  
      currentclusters

This is the meat of the program.  The two main versions of clusterData (other than the "empty points" one) call updateclusters as the means of allocating each new point.

updateclusters::usage = 
      "updateclusters is an auxiliary function that determines which \
cluster to place a new point into."

\(\(updateclusters[currentclusters : {__?MatrixQ}, newpoint_?VectorQ, 
          threshold_Real] /; 
	
        \((And\ @@\ 
            \((\ Thread[
                \((\(\((\(Dimensions[#]\)[[2
                            ]])\)\ &\)\ /@\ 
                      currentclusters)\) == Length[newpoint]\ ])\))\) := 
	
      With[{centroids = 
            Sqrt[Apply[Plus, \ 
                \((\(\((\((First[#] - newpoint)\)^2)\)&\)\ /@currentclusters)
                  \), {1}]]}, 
	
        With[{closest = 
              \(Flatten[Position[centroids, Min[centroids]]]
                  \)[[1]]}, 
		
          If[centroids[[closest]] < 
              threshold, 
			
            If[\(Dimensions[currentclusters]\)[[1
                  ]] == 1, 
				
              Insert[currentclusters, {newpoint}, {\(-1\)}], 
              Insert[currentclusters, 
                newpoint, {closest, 
                  Length[currentclusters[[closest
                        ]]] + 1}]], 
            Join[currentclusters, {{newpoint}}]]\ ]]\ \)\)

These are two information statistics that are intended to help determine the best number of clusters. 
clusterLambdaStatistic is the ratio of the determinant of the matrix of sums of sums of squares and crossproducts of each cluster, to the determinant of the matrix of the sums of squares and cross products for the whole dataset.

Think of this as the ratio of the sum of the variations within each cluster, to the total variation in the data set.  Smaller numbers are better.

MarriotClusterStatistic trades off parsimony for explanatory power, being the product of the square of the number of clusters, and the determinant of the matrix of sums of sums of squares and crossproducts of each cluster.

Think of this as the product of the total within-cluster variation and the square of the number of clusters.  The more clusters, the less variation within each of them, but this is traded off against the number of clusters.  Again, the smaller the value of the statistic, the better.

\(clusterLambdaStatistic[clusters : {__?MatrixQ}] /; 
        \((Equal\ @@\ 
            \((\(\((\(Dimensions[#]\)[[2
                      ]])\)\ &\)\ /@\ clusters)\))\) := 

      	With[{joined = Join@@\ clusters}, 
		
        Det[Plus\ @@
              \((\(Apply[Plus, 
                      Apply\ [Plus, Outer[Times, Transpose[#], #], {1}], {1}]&
                    \)\ /@\ clusters)\)]/
				
          Det[Apply[Plus, 
              Apply\ [Plus, 
                Outer[Times, Transpose[\((joined)\)], \((joined)\)], {1}], {
                1}]]\ ]\)

\(MarriotClusterStatistic[clusters : {__?MatrixQ}] /; 
        \((Equal\ @@\ 
            \((\(\((\(Dimensions[#]\)[[2
                      ]])\)\ &\)\ /@\ clusters)\))\) := 

      			Length[clusters]^2*
        Det[Plus\ @@
            \((\(Apply[Plus, 
                    Apply\ [Plus, Outer[Times, Transpose[#], #], {1}], {1}]&
                  \)\ /@\ clusters)\)]\)