# v_entropy

## PURPOSE

V_ENTROPY calculates the v_entropy of discrete and sampled continuous distributions H=(P,DIM,COND,ARG,STEP)

## SYNOPSIS

function h=v_entropy(p,dim,cond,arg,step)

## DESCRIPTION

    V_ENTROPY calculates the v_entropy of discrete and sampled continuous distributions H=(P,DIM,COND,ARG,STEP)

      Inputs:  P        is a vector or matrix of probabilities - one dimension per variable
               DIM      lists dimensions along which to evaluate the v_entropy [default: 1st non singleton dimension]
               COND     lists dimensions to use as conditional variables [default - none]
               ARG      lists dimensions to use as parameters in the ouput [default - none]
               STEP     for continuous distributions STEP gives the sample increment for each dimension of P
                        if STEP is a scalar, the increment is assumed to be the same for each dimension

     Outputs:  H        is the v_entropy. It will have the same number of dimensions as the length of the ARG input.
                        If the STEP argument is specified then this will be the differential v_entropy.

     Example: Suppose P(W,X,Y,Z) represents the joint probability of four correlated random variables

                   (a) H(W,X,Y,Z) = v_entropy(P,[1 2 3 4]).
                   (b) H(W) = v_entropy(P), or equivalently v_entropy(P,1)
                   (c) H(W,Z | X,Y) = v_entropy(P,[1 4],[2 3])
                   (d) H(W | X, Z=z) = v_entropy(P,1,2,4); this is a function of z and will be a column vector

     As a special case, if the dimensions included in DIM are all singletons, the entries in P are treated
     as Bernoulli variable probabilities.

## CROSS-REFERENCE INFORMATION

This function calls:

- v_entropy V_ENTROPY calculates the v_entropy of discrete and sampled continuous distributions H=(P,DIM,COND,ARG,STEP)

This function is called by:

- v_entropy V_ENTROPY calculates the v_entropy of discrete and sampled continuous distributions H=(P,DIM,COND,ARG,STEP)

## SOURCE CODE

    0001 function h=v_entropy(p,dim,cond,arg,step)
    0002 %V_ENTROPY calculates the v_entropy of discrete and sampled continuous distributions H=(P,DIM,COND,ARG,STEP)
    0003 %
    0004 %  Inputs:  P        is a vector or matrix of probabilities - one dimension per variable
    0005 %           DIM      lists dimensions along which to evaluate the v_entropy [default: 1st non singleton dimension]
    0006 %           COND     lists dimensions to use as conditional variables [default - none]
    0007 %           ARG      lists dimensions to use as parameters in the ouput [default - none]
    0008 %           STEP     for continuous distributions STEP gives the sample increment for each dimension of P
    0009 %                    if STEP is a scalar, the increment is assumed to be the same for each dimension
    0010 %
    0011 % Outputs:  H        is the v_entropy. It will have the same number of dimensions as the length of the ARG input.
    0012 %                    If the STEP argument is specified then this will be the differential v_entropy.
    0013 %
    0014 % Example: Suppose P(W,X,Y,Z) represents the joint probability of four correlated random variables
    0015 %
    0016 %               (a) H(W,X,Y,Z) = v_entropy(P,[1 2 3 4]).
    0017 %               (b) H(W) = v_entropy(P), or equivalently v_entropy(P,1)
    0018 %               (c) H(W,Z | X,Y) = v_entropy(P,[1 4],[2 3])
    0019 %               (d) H(W | X, Z=z) = v_entropy(P,1,2,4); this is a function of z and will be a column vector
    0020 %
    0021 % As a special case, if the dimensions included in DIM are all singletons, the entries in P are treated
    0022 % as Bernoulli variable probabilities.
    0023
    0024 %       Copyright (C) Mike Brookes 2006
    0025 %      Version: $Id: v_entropy.m 10865 2018-09-21 17:22:45Z dmb $
    0026 %
    0027 %   VOICEBOX is a MATLAB toolbox for speech processing.
    0028 %   Home page: http://www.ee.ic.ac.uk/hp/staff/dmb/voicebox/voicebox.html
    0029 %
    0030 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    0031 %   This program is free software; you can redistribute it and/or modify
    0032 %   it under the terms of the GNU Lesser General Public License as published by
    0033 %   the Free Software Foundation; either version 3 of the License, or
    0034 %   (at your option) any later version.
    0035 %
    0036 %   This program is distributed in the hope that it will be useful,
    0037 %   but WITHOUT ANY WARRANTY; without even the implied warranty of
    0038 %   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    0039 %   GNU Lesser General Public License for more details.
    0040 %
    0041 %   You can obtain a copy of the GNU Lesser General Public License from
    0042 %   https://www.gnu.org/licenses/ .
    0043 %
    0044 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    0045
    0046 if nargin<5
    0047     stp=zeros(ndims(p),1);
    0048 else
    0049     stp=repmat(step(1),ndims(p),1);
    0050     stp(1:length(step))=step(:);
    0051     stp=log2(stp);
    0052 end
    0053 if nargin<4
    0054     arg=[];
    0055 else
    0056     arg=arg(arg>0);
    0057 end
    0058 if nargin<3
    0059     cond=[];
    0060 else
    0061     cond=cond(cond>0);
    0062 end
    0063 if ~length(cond)
    0064     s=size(p);
    0065     if nargin<2
    0066         dim=find(s>=min(2,max(s)));
    0067         dim=dim(1);
    0068     else
    0069         dim=dim(dim>0);
    0070     end
    0071     st=prod(s);
    0072     sd=prod(s(dim));
    0073     sa=prod(s(arg));
    0074     marg=1:length(s);
    0075     marg(arg)=0;
    0076     marg(dim)=0;
    0077     marg=marg(marg>0);
    0078     sm=st/sd/sa;
    0079     if sm>1
    0080         ip=[arg dim(:)' marg(:)'];
    0081         sp=[s([arg dim(:)']) prod(s(marg))];
    0082         q=sum(reshape(permute(p,[arg(:)' dim(:)' marg]),sa,sd,sm),3);
    0083     else
    0084         q=reshape(permute(p,[arg(:)' dim(:)' marg]),sa,sd);
    0085     end
    0086     if sd==1
    0087         h=-log2(q+(q==0)).*q-log2(1-q+(q==1)).*(1-q);   % special treatment for bernoulli variables
    0088     else
    0089         sq=sum(q,2);
    0090         h=sum(-log2(q+(q==0)).*q,2)./sq+log2(sq);
    0091     end
    0092     if length(arg)>1
    0093         h=reshape(h,s(arg));
    0094     end
    0095     h=h+sum(stp(dim));
    0096 else
    0097     % we could probably make this more efficient by avoiding the recursive call
    0098     h=v_entropy(p,[dim(:); cond(:)],0,arg)-v_entropy(p,cond,0,arg);
    0099 end