hi,

"Bald Eagle" <cre### [at] netscapenet> wrote:
> "jr" <cre### [at] gmailcom> wrote:
> > > it's been .. quiet.
>
> Well, been busy with RL, reading, re-reading, watching vids, filtering, taking
> notes, processing, etc.
>
> I think I have enough of an understanding of what goes on and what to do with
> it, that it ought to be a viable method.
>
> Will start a new thread in pbi so I can show results.

looks great.

> Thanks for the line of code - that will be the next step after some more
> tinkering:  packaging it up into an easy to use utility.

after posting it occurred to me that you might want your data files resemble the
matrix, rather than having a value per line.  :-)  assuming a justified (with
space chars) layout like:
2
123.0    4.56
  7.89  -2.0

you can use standard *NIX text processing tools to "massage" the data:
$ tr -s ' ' '\n' < myfile | grep -v -e ^$ | ./a.out

in English, 'tr' ("translate") converts spaces to newlines, the '-s' option
"squeezes" any number of consecutive chars into one, the grep looks for lines
which do not match an empty line.

looking forward to more on svd.  is it useful only for (vaguely)
spherical/elliptical shapes?


regards, jr.

Post a reply to this message

"jr" <cre### [at] gmailcom> wrote:

> looks great.
>
> > Thanks for the line of code - that will be the next step after some more
> > tinkering:  packaging it up into an easy to use utility.
>
> after posting it occurred to me that you might want your data files resemble the
> matrix, rather than having a value per line.

Yes, that's what I was planning on.

> looking forward to more on svd.  is it useful only for (vaguely)
> spherical/elliptical shapes?

Nah.  Anything that doesn't have a symmetric distribution.

so I'd say that off-axis linear data can be oriented,

2D data can be aligned so that the most elongated part is along one axis, and
the rest of the data perpendicular to that gets rotated to be in the cardinal
plane,

and the ellipsoid is just the easy 3D version of that, but any set of vectors
could be used.

Hopefully some of the finer points become clearer in the near future.

Looking at what I just posted, and how I've noticed changes in the SVD output
with different data, maybe all I need for a better alignment is more points ...

Post a reply to this message

hi,

"Bald Eagle" <cre### [at] netscapenet> wrote:
> "jr" <cre### [at] gmailcom> wrote:
> > looks great.
> >
> > > Thanks for the line of code - that will be the next step after some more
> > > tinkering:  packaging it up into an easy to use utility.
> >
> > after posting it occurred to me that you might want your data files resemble the
> > matrix, rather than having a value per line.
>
> Yes, that's what I was planning on.

after posting _that_ I remembered you mentioning your being new to Linux etc in
another thread.  so some more on command lines etc below.  (I really feel old --
not so many years ago I'd have covered the salient points in a single reply.
thinking is getting harder.  :-()

> > looking forward to more on svd.  is it useful only for (vaguely)
> > spherical/elliptical shapes?
>
> Nah.  Anything that doesn't have a symmetric distribution.
> so I'd say that off-axis linear data can be oriented,

and can be just a set of data points, ie no "shape" required?  powerful.

> 2D data can be aligned so that the most elongated part is along one axis, and
> the rest of the data perpendicular to that gets rotated to be in the cardinal
> plane,

interesting too but cannot visualise use cases.  when (what kind of data) would
you use that?

> and the ellipsoid is just the easy 3D version of that, but any set of vectors
> could be used.
>
> Hopefully some of the finer points become clearer in the near future.
>
> Looking at what I just posted, and how I've noticed changes in the SVD output
> with different data, maybe all I need for a better alignment is more points ...

testing, testing..  :-)

so, "..you might want your data files resemble the matrix, rather than having a
value per line."

and also, conceivably :-), you might want to document the data.  that
command-line for cleaning ("re-formatting") the data -- no one likes to type
stuff like it all the time.  but there's no need since the tools are all
provided.

typically, a '#' in the first column of a line starts a comment, so the matrix
data file could be:

  # mydata - from some/url.
  2
  123.0    4.56
  7.89    -2.0

the shell (BASH) lets you create functions[*], though they do not have to return
a value, which in use are indistinguishable from "real" utilities.  they can be
typed in on the command-line or sourced from file.  my usual method (ymmv) is to
create a function in a scratch file, test it "to death" ;-), then transfer it to
my '~/.bash_functions' file (think personal library); sourced from the
'~/.bashrc', so gets loaded automatically.

[*] an 'alias' would be no good since you can only append to them, the
alternative would be a "proper" shell script.

for the svd invocation, presuming that you'll go with the above file format (can
have more than one comment line, of course), one way of writing that function is
(fingers crossed wrt typos):

  function fnsvd () {
    if [ 1 -ne $# ]; then
      echo usage: $FUNCNAME matrix_data
    else
      grep -v -e ^# | tr -s ' ' '\n' < $1 | grep -v -e ^$ | /path/to/svd/a.out
    fi
  }

same as before but removing the comments first.  if you're in the (bad, imo)
habit of having spaces in file names, you'd need to put the '$1' parameter in
double quotes.  _is_ extra work, but you only type it once.  wrt the name, I
generally use a 'fn' prefix to remind self.

once that is available in the shell, ie has been 'source'd, you can simply:

  $ fnsvd mydata

the last line in my '~/.bashrc' reads:

  if [ -f ~/.bash_functions ]; then source ~/.bash_functions; fi

note that I tend to write out 'source', usually you'll see people use the
shorthand, a period/dot.  hope that helps in some way, and, yeah, sorry about
the .. noise.


regards, jr.

Post a reply to this message

"jr" <cre### [at] gmailcom> wrote:
> ...
> (fingers crossed wrt typos):
>
>   function fnsvd () {
>     if [ 1 -ne $# ]; then
>       echo usage: $FUNCNAME matrix_data
>     else
>       grep -v -e ^# | tr -s ' ' '\n' < $1 | grep -v -e ^$ | /path/to/svd/a.out
>     fi
>   }

</sigh>  blindly copying + pasting does that.  :-(  corrected:

  grep -v -e ^# $1 | tr -s ' ' '\n' | grep -v -e ^$ | /path/to/svd/a.out


regards, jr.

Post a reply to this message

"jr" <cre### [at] gmailcom> wrote:

> after posting _that_ I remembered you mentioning your being new to Linux etc in
> another thread.  so some more on command lines etc below.  (I really feel old -

I was new 20 years ago, I just haven't been involved in administration of the
NMR computers or had the SGI O2 to play with for about a decade.  So every
little bit helps, whether it's something I "know" or not.



> and can be just a set of data points, ie no "shape" required?  powerful.

Correct.   Data is data is data.  It doesn't have to be geometric at all.
It could be correlating customer purchases with zip codes or what movie watchers
might also like to watch, or what their most likely preferred theatre snack is,
or just about anything that you can think of.


> > 2D data can be aligned so that the most elongated part is along one axis, and
> > the rest of the data perpendicular to that gets rotated to be in the cardinal
> > plane,
>
> interesting too but cannot visualise use cases.  when (what kind of data) would
> you use that?

Just search "application of singular value decomposition to ...."  and it goes
on and on and on....


> testing, testing..  :-)

Yeah.   I upped my test case to 400 points, switched over from the "narrow"
version to the "wide" version, found that all of that introduces some problems,
learned a bit more (logical in retrospect) about the resulting matrices, and now
am awaiting some replies from hopefully helpful experts On The Internet.

I used to write shell scripts, and do a bunch of regular expression stuff, but I
have to get back into that - time, energy, focus, round tuits allowing....

Good progress so far, just have to make it past the usual frustrating roadblocks
to reach the end....

Post a reply to this message

Just an interesting aside:

https://blogs.mathworks.com/cleve/2012/12/10/1976-matrix-singular-value-decomposition-film/

Post a reply to this message

hi,

"Bald Eagle" <cre### [at] netscapenet> wrote:
> Just an interesting aside:
>
>
https://blogs.mathworks.com/cleve/2012/12/10/1976-matrix-singular-value-decomposition-film/

thank you for this link.  watching the film helped somewhat.  (shades of Tron
:-))


regards, jr.

Post a reply to this message

"jr" <cre### [at] gmailcom> wrote:

> thank you for this link.  watching the film helped somewhat.

You, Sir, are hilarious.
That thing didn't help me a single bit.  :D

Try this.
https://towardsdatascience.com/svd-8c2f72e264f

There are dozens of other pretty decent summaries / explanations.

Now that I have a functional reorientation, I'm waiting on some feedback from
someone, anyone as to a last bit of the puzzle (unconnected to the present
application).

Also going to try to get some c++ code written, and then hopefully the SVD will
be able to be integrated into a easy-to-use POV-Ray workflow.

If THAT works, then I have some other ideas for what to try it on.

Post a reply to this message

So, since I don't code in c++, I'm kinda coding by imitation and trial and
error.

I get the current small bit of test code to compile, but when I run it with my
dataset: "400" followed by 1200 numbers "value\n"

#write (Output, NumPoints, "\n")
 #write (Output, P[PonS].x, "\n")
 #write (Output, P[PonS].y, "\n")
 #write (Output, P[PonS].z, "\n")

I get a segmentation fault.


template<typename Arg = cpp_bin_float_4096>
void generate_matrix(std::vector<std::vector<cpp_bin_float_4096> >& \
 matrix, std::size_t rows, std::size_t cols)

{
    std::cin >> rows;
 std::srand((unsigned int)std::time(nullptr)); matrix.resize(rows);
 for (std::size_t row = 0; row < matrix.size(); row++)
 {
  matrix[row].resize(3);
  for (std::size_t col = 0; col < 3; col++)
   //std::cin >> ignore(100, '\n') >> matrix[rows][cols];
            std::cin >> matrix[rows][cols];
 }
    std::cout << "Data imported. \n\n";
}




int main(int argc, char* argv[])
{
 std::size_t matrix_size = 0;
 std::vector<std::vector<cpp_bin_float_4096> > u, v;
 std::vector<std::vector<cpp_bin_float_4096> > matrix, s;
 std::cout << "Importing data into matrix . . . \n\n";


  generate_matrix(matrix, matrix_size, matrix_size);

  std::cout << "\nA = \n"; print_matrix(matrix);

  //svd(matrix, s, u, v);

  //std::cout << "\nS = \n"; print_matrix(s);
  //std::cout << "\nU = \n"; print_matrix(u);
  //std::cout << "\nV = \n"; print_matrix(v);

 std::cin.get();
 //std::cin.get();

 return 0;
}

Post a reply to this message

hi,

"Bald Eagle" <cre### [at] netscapenet> wrote:
> So, since I don't code in c++, I'm kinda coding by imitation and trial and
> error.
>
> I get the current small bit of test code to compile, but when I run it with my
> dataset: "400" followed by 1200 numbers "value\n"
>
> #write (Output, NumPoints, "\n")
>  #write (Output, P[PonS].x, "\n")
>  #write (Output, P[PonS].y, "\n")
>  #write (Output, P[PonS].z, "\n")
>
> I get a segmentation fault.

> ...

please post a data file.  when you say "400", are you trying to run a 400x400
matrix?  not sure but you *might* hit the limits -- the code's recursive, no?

I'll monkey my version + try the data, in coming days.


regards, jr.

Post a reply to this message