Too Many Cooks Spoil the Broth

Well dear readers, I have come to say that I have decided to end my Help Wanted search. I have assembled a small team of capable people, and together we will find our fortune. I am amazed at how much new territory awaits us, and portfolio theory indeed holds many exciting places for us to explore. I can only hope our bravery is rewarded with success.

My financial postings here will now be extremely limited; furthermore I must protect the privacy and hard work of my teammates as well by keeping private our team efforts.

Help Wanted, Part 2

Since it has been many months since my previous call for Help Wanted, I wanted to post and update to describe my most recent status.

At this point, I am only considering interested persons who have direct experience with all/most of these subjects:

1. C++
2. Information theory, or doctorate-level finance or signal processing.
3. Weekly time to devote to this project.

Ordinarily I would post a summary of my progress, but at this point I have decided to cease all future posts about my personal trading efforts. I have reached that (exciting) point in my development where my ideas can be commercialized, and disclosing confidential information cannot help me from this point on. I am still open to adding partners, but given my progress over the last few months, the bar is high. I may make a post about trading-related subjects not directly related to my ongoing efforts -- there is still a tremendous amount of interesting trading theories in the world, and they all deserve some amount of appreciation.

I am grateful to all the other trading authors who take time to post their experiences online. I derive my inspiration from them, and only wanted to give back as best I could. I think Cover's Rebalanced Portfolio concepts are very exciting, and I encourage all others to look into it. Do not be discouraged by the math; everything is understandable if you can visualize it the right way. There are plenty of resources out there for you on the web to read, and the opportunity for learning is infinite (myself included).

The Woes of Backtesting

I have been secretly toiling away in my Mad Scientist Lair, perfecting my portfolio auto-trader. One of the first things which really slowed me down was the distinct lack of good historical data to test with. I have collected quite a list of historical Forex data sites, including just about everything you can possibly find with Google. Some claim to have 1-minute periods going back before Y2K. While it's true that the data does indeed go back that far, for the most part you are lucky if the data is contiguous.

My hard lesson is that "free" Forex data is about as good as it's price. Every single one I can find offering intraday periods for free has significant gaps in the data. It is important to emphasize that these gaps are not from weekends either; those are easy to accommodate (and of course irrelevant to backtesting). I then tried the Humpty-Dumpty solution by combining the available data to form a composite history. Imagine my disappointment when I found two very notable sources (and popular among Forex backtesters) differ by more than 10 pips, consistently, across hundreds of periods. This difference was not constant either over time. I am shocked to see how poor the quality of historical data is. Even if I were doing a naive trading system using MetaTrader I would not rely on any of that stuff.

So the bad news is that I need to go out of pocket to get the data I need for testing. In the long term, this data will become another asset for me. At least I can count on having data without missing periods, bad ticks, or other garbage to be cleaned up.

Help Wanted

I have been resisting doing this, but I'm biting the bullet and making a public post here.

I know my blog doesn't have much traffic, and I've not been posting much lately. However, I have decided to take on a partner or two in my code development. I am interested in working with someone smart and motivated on my Cover's Portfolio algorithm. Any approved person(s) will get full access to all the source code, and lots of input on tweaking it.

Here are my minimum requirements:

1. 5+ years writing C++ code
2. Familiar with some form of trading, either day trading, gambling theory, or financial engineering. I have some flexibility here in background requirements.
3. Ability and time to work at least 5+ hours a week on developing and testing the code.
4. Ability to read and interpret technical papers and mathematical notation.
   

I am not considering anyone who cannot meet all these guidelines; I would rather have no development partners than ones who will burden me with basic questions. I have done a lot of heavy lifting already, and I intend to make production-ready the optimized Monte-Carlo version of Cover's algorithm. Initially, the only code I need to develop is a standalone Cover-MC solver. Eventually, this codebase will expand to include:

• NVidia CUDA implementation
  
• backtesting infrastructure
  
• auto-trading code

If you are interested, please click on my profile link and send me email via that method. I will respond within about 48 hours.

Making the Most of Porfolios

It's been quite awhile since I last posted on Portfolio Theory. I have been running some tests recently, and confirmed some of the suspicions I have had for at least a year. I am running some backtests on Portfolio algorithms, and I have seen some powerful evidence of Cover's "Side Information" optimization. Side Information is nothing more than subdividing a time series into N separate portfolio domains, using some indicator or derived value. The trick, of course, is to choose an optimal filter by which to create each subdomain.

If you look at how Porfolio Rebalancing works, it should become clear that it works best when individual securities have long term trends. It also works well when the securities are uncorrelated, and even some negatively correlated elements. Negative correlation works well to offset down-trending securities; keeping a small weighting in anti-trending securities preserves capital during drawdowns. Uncorrelated securities help by smoothing out the trend-related swings in capital. Have a large number of uncorrelated securities is highly desireable in any Rebalanced Portfolio.

Those principles notwithstanding, I have a great example of how dividing a time-series (a.k.a. price-history) into domains can greatly impact profitability. Shown below is a picture of a pretty nice stock history (actually a currency, but that doesn't matter):Clearly this is a great performing security. One excellent question I ask is: Can we extract profits from this uptrend by filtering out those periods which are negative? The answer of course, is yes we can.

By using a novel filtering technique, the original time series can be divided according to Cover's Side Information. Using my latest version of this filter, here is a subdomain created which predicts the largest drops in value:
Clearly this filter isn't perfect, but it does a pretty consistent job of extracting a downtrend from the original time series, which is heavily upward biased. The fact that a reasonable downtrend can be extracted yeilds huge potential to optimize a separate set of portfolio weights tailored to each domain. For the icing on the cake, check out the positive filtered domain:
Not only does the positive-filtered achieve greater growth, but actually has lower volatility and higher Sharpe Ratio. These characteristics are extremely desireable for excellent capital growth.

I withhold the equity curve from this system in order to preserve some confidentiality of my IP, but also because my domain filtering algorithm is still under development.

The Skunkworks are Busy

So I have several really cool things that are happening in my life, but I can't really post about them. My biggest liability is that this blog is public, and visible to pretty much anyone. There are many new developments in my business environment and personal life that are very exciting, but in each case there are some specific reasons it needs to stay under wraps. Very frustrating.

In the more mundane domain, I am very much enjoying my new Samsung 46" LCD TV, and also my new Canon A590is camera. They were gifts to myself for saving money this year and working hard. I'm sure it will be followed up by an increase in my 401k deposits, and more savings increases.

LinkedIn : The Good and the Bad

I have been a big fan of the LinkedIn professional network. I don't consider it a social one, because it caters to professionals and formal organizations (e.g. alumni groups, professional bodies, non-virtual entities). I have met some very powerful people in the real world because they found me via LinkedIn. I consider this fundamentally more useful than "social" sites like MySpace and Facebook, because those sites are primarily geared toward entertainment and purely social activities. LinkedIn can be a powerful tool for networking with influential people, if you use it right.

The trick of course is to be responsible about who you invite into your network. It pays to seek out people you know who either have powerful contacts, or who are experienced professionals. By prioritizing these contacts, you can actually gain a way to contact people who can advance your career. I was able to add several Venture Capitalists to my network this way, as well as some technology company founders. These are people who are potentially capable of helping me in the future, should I strike out on my own.

I am surprised by a general "promiscuity" in adding literally anyone into the 1st level network. I came across these guys: http://www.toplinked.com/. They seem to have a very naieve approach to networking: add anyone. The problem with this strategy is several fold. First, there is no "quality control". You are far more likely to find useless contacts in this network, which you won't know if there is no other way to cross-reference them. Second, there is no real advantage for experienced/networked people to join. If anything, people of high network caliber are dis-incented to join, because they don't need contacts who have little professional influence. Third, it neutralizes the benefits of networking: no-one can vouch for another, and it becomes essentially like having a phone-book full of names. It takes an idea that is inherently modular, and flattens it. I don't see how that is useful.