Home > v_mfiles > v_entropy.m

# v_entropy

## PURPOSE V_ENTROPY calculates the v_entropy of discrete and sampled continuous distributions H=(P,DIM,COND,ARG,STEP)

## SYNOPSIS function h=v_entropy(p,dim,cond,arg,step)

## DESCRIPTION ```V_ENTROPY calculates the v_entropy of discrete and sampled continuous distributions H=(P,DIM,COND,ARG,STEP)

Inputs:  P        is a vector or matrix of probabilities - one dimension per variable
DIM      lists dimensions along which to evaluate the v_entropy [default: 1st non singleton dimension]
COND     lists dimensions to use as conditional variables [default - none]
ARG      lists dimensions to use as parameters in the ouput [default - none]
STEP     for continuous distributions STEP gives the sample increment for each dimension of P
if STEP is a scalar, the increment is assumed to be the same for each dimension

Outputs:  H        is the v_entropy. It will have the same number of dimensions as the length of the ARG input.
If the STEP argument is specified then this will be the differential v_entropy.

Example: Suppose P(W,X,Y,Z) represents the joint probability of four correlated random variables

(a) H(W,X,Y,Z) = v_entropy(P,[1 2 3 4]).
(b) H(W) = v_entropy(P), or equivalently v_entropy(P,1)
(c) H(W,Z | X,Y) = v_entropy(P,[1 4],[2 3])
(d) H(W | X, Z=z) = v_entropy(P,1,2,4); this is a function of z and will be a column vector

As a special case, if the dimensions included in DIM are all singletons, the entries in P are treated
as Bernoulli variable probabilities.```

## CROSS-REFERENCE INFORMATION This function calls:
• v_entropy V_ENTROPY calculates the v_entropy of discrete and sampled continuous distributions H=(P,DIM,COND,ARG,STEP)
This function is called by:
• v_entropy V_ENTROPY calculates the v_entropy of discrete and sampled continuous distributions H=(P,DIM,COND,ARG,STEP)

## SOURCE CODE ```0001 function h=v_entropy(p,dim,cond,arg,step)
0002 %V_ENTROPY calculates the v_entropy of discrete and sampled continuous distributions H=(P,DIM,COND,ARG,STEP)
0003 %
0004 %  Inputs:  P        is a vector or matrix of probabilities - one dimension per variable
0005 %           DIM      lists dimensions along which to evaluate the v_entropy [default: 1st non singleton dimension]
0006 %           COND     lists dimensions to use as conditional variables [default - none]
0007 %           ARG      lists dimensions to use as parameters in the ouput [default - none]
0008 %           STEP     for continuous distributions STEP gives the sample increment for each dimension of P
0009 %                    if STEP is a scalar, the increment is assumed to be the same for each dimension
0010 %
0011 % Outputs:  H        is the v_entropy. It will have the same number of dimensions as the length of the ARG input.
0012 %                    If the STEP argument is specified then this will be the differential v_entropy.
0013 %
0014 % Example: Suppose P(W,X,Y,Z) represents the joint probability of four correlated random variables
0015 %
0016 %               (a) H(W,X,Y,Z) = v_entropy(P,[1 2 3 4]).
0017 %               (b) H(W) = v_entropy(P), or equivalently v_entropy(P,1)
0018 %               (c) H(W,Z | X,Y) = v_entropy(P,[1 4],[2 3])
0019 %               (d) H(W | X, Z=z) = v_entropy(P,1,2,4); this is a function of z and will be a column vector
0020 %
0021 % As a special case, if the dimensions included in DIM are all singletons, the entries in P are treated
0022 % as Bernoulli variable probabilities.
0023
0024 %       Copyright (C) Mike Brookes 2006
0025 %      Version: \$Id: v_entropy.m 10865 2018-09-21 17:22:45Z dmb \$
0026 %
0027 %   VOICEBOX is a MATLAB toolbox for speech processing.
0029 %
0030 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
0031 %   This program is free software; you can redistribute it and/or modify
0032 %   it under the terms of the GNU General Public License as published by
0033 %   the Free Software Foundation; either version 2 of the License, or
0034 %   (at your option) any later version.
0035 %
0036 %   This program is distributed in the hope that it will be useful,
0037 %   but WITHOUT ANY WARRANTY; without even the implied warranty of
0038 %   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
0039 %   GNU General Public License for more details.
0040 %
0041 %   You can obtain a copy of the GNU General Public License from
0042 %   http://www.gnu.org/copyleft/gpl.html or by writing to
0043 %   Free Software Foundation, Inc.,675 Mass Ave, Cambridge, MA 02139, USA.
0044 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
0045
0046 if nargin<5
0047     stp=zeros(ndims(p),1);
0048 else
0049     stp=repmat(step(1),ndims(p),1);
0050     stp(1:length(step))=step(:);
0051     stp=log2(stp);
0052 end
0053 if nargin<4
0054     arg=[];
0055 else
0056     arg=arg(arg>0);
0057 end
0058 if nargin<3
0059     cond=[];
0060 else
0061     cond=cond(cond>0);
0062 end
0063 if ~length(cond)
0064     s=size(p);
0065     if nargin<2
0066         dim=find(s>=min(2,max(s)));
0067         dim=dim(1);
0068     else
0069         dim=dim(dim>0);
0070     end
0071     st=prod(s);
0072     sd=prod(s(dim));
0073     sa=prod(s(arg));
0074     marg=1:length(s);
0075     marg(arg)=0;
0076     marg(dim)=0;
0077     marg=marg(marg>0);
0078     sm=st/sd/sa;
0079     if sm>1
0080         ip=[arg dim(:)' marg(:)'];
0081         sp=[s([arg dim(:)']) prod(s(marg))];
0082         q=sum(reshape(permute(p,[arg(:)' dim(:)' marg]),sa,sd,sm),3);
0083     else
0084         q=reshape(permute(p,[arg(:)' dim(:)' marg]),sa,sd);
0085     end
0086     if sd==1
0087         h=-log2(q+(q==0)).*q-log2(1-q+(q==1)).*(1-q);   % special treatment for bernoulli variables
0088     else
0089         sq=sum(q,2);
0090         h=sum(-log2(q+(q==0)).*q,2)./sq+log2(sq);
0091     end
0092     if length(arg)>1
0093         h=reshape(h,s(arg));
0094     end
0095     h=h+sum(stp(dim));
0096 else
0097     % we could probably make this more efficient by avoiding the recursive call
0098     h=v_entropy(p,[dim(:); cond(:)],0,arg)-v_entropy(p,cond,0,arg);
0099 end```

Generated by m2html © 2003