Tag Archives: Palantir Government

September 24, 2016 · 14:51

Room for Requirement

I looked at a few issues 3 days ago. I voiced them in my blog ‘The Right Tone‘ (at https://lawlordtobe.com/2016/09/21/the-right-tone/), one day later we see ‘MI6 to recruit hundreds more staff in response to digital technology‘ (at https://www.theguardian.com/uk-news/2016/sep/21/mi6-recruit-digital-internet-social-media), what is interesting here is the quote “The information revolution fundamentally changes our operating environment. In five years’ time there will be two sorts of intelligence services: those that understand this fact and have prospered, and those that don’t and haven’t. And I’m determined that MI6 will be in the former category“, now compare it to the statement I had made one day earlier “The intelligence community needs a new kind of technological solution that is set on a different premise. Not just who is possibly guilty, but the ability of aggregation of data flags, where not to waste resources“, which is just one of many sides needed. Alex Younger also said: “Our opponents, who are unconstrained by conditions of lawfulness or proportionality, can use these capabilities to gain increasing visibility of our activities which means that we have to completely change the way that we do stuff”, I reckon the American expression: ‘He ain’t whistling Dixie‘ applies.

You see, the issue goes deeper than mere approach, the issue at hand is technology. The technology needs to change and the way data is handled requires evolution. I have been in the data field since the late 80’s and this field hasn’t changed too much. Let’s face it, parsing data is not a field that has seen too much evolving, for the mere reason that parsing is parsing and that is all about speed. So to put it on a different vehicle. We are entering an age where the intelligence community is about the haulage of data, yet in all this, it is the container itself that grows whilst the haulage is on route. So we need to find alternative matters to deal with the container content whilst on route.

Consider the data premise: ‘If data that needs processing grows by 500 man years of work on a daily basis‘, we have to either process smarter, create a more solutions to process, be smarter on what and how to process, or change the premise of time. Now let’s take another look. For this let’s take a look at a game, the game ‘No Man’s Sky’. This is not about gaming, but about the design. For decades games were drawn and loaded. A map, with its data map (quite literally so). Usually the largest part of the entire game. 11 people decided to use a formula to procedurally generate 18 quintillion planets. They created a formula to map the universe with planets, planet sized. This has never been done before! This is an important part. He turned it all around and moreover, he is sitting on a solution that is worth millions, it could even be worth billions. The reason to use this example is because games are usually the first field where the edge of hardware options are surpassed, broken and redesigned (and there is more at the end of this article). Issues that require addressing in the data field too.

Yet what approach would work?

That is pretty much the ‎£1 billion question. Consider the following situation: Data is being collected non-stop, minute by minute. Set into all kinds of data repositories. Now let’s have a fictive case. The chatter gives that in 72 hours an attack will take place, somewhere in the UK. It gives us the premise:

Who
Where
How

Now consider the data. If we have all the phone records, who has been contacting who, through what methods and when? You see, it isn’t about the data, it is about linking collections from different sources and finding the right needle, that whilst the location, shape and size of the haystack are an unknown. Now, let’s say that the terrorist was really stupid and that number is known. So now we have to get a list of all the numbers that this phone had dialled. Then we get the task of linking the information on these people (when they are not pre-paid or burner phones). Next is the task of getting a profile, contacts, places, and other information. The list goes on and the complexity isn’t just the data, the fact that actual terrorists are not dumb and usually massively paranoid, so there is a limit to the data available.

Now what if this was not reactive, but proactive?

What if the data from all the sources could be linked? Social media, e-mail, connections, forums and that is just the directly stored data. When we add mobile devices, Smartphones, tablets and laptops, there is a massive amount of additional data that becomes available and the amount of data from those sources are growing at an alarming rate. The challenge is to correctly link the data from sources, with added data sources that contain aggregated data. So, how do you connect these different sources? I am not talking about the usage, it is about the impaired data on different foundations with no way to tell whether pairing leads to anything. For this I need to head towards a 2012 article by Hsinchun Chen (attached at end), Apart from the clarity that we see in the BI&A overview (Evolution, Application and Emerging Research), the interesting part that even when we just look at it from a BI point of view, we see two paths missing. That is, they seem to be missing now, if we look back to 2010-2011, the fact that Google and Apple grew a market in excess of 100% quarter on quarter was not to be anticipated to that degree. The image on page 1167 has Big Data Analytics and Mobile Analytics, yet Predictive Interactivity and Mobile Predictive Analytics were not part of the map, even though the growth of Predictive Analytics have been part of BI from 2005 onwards. Just in case you were wondering, I did not change subject, the software need that part of the Intelligence world uses comes from the business part. A company usually sees a lot more business from 23 million global companies than it gets from 23 intelligence agencies. The BI part is often much easier to see and track whilst both needs are served. We see a shift of it all when we look at the table on page 1169. BI&A 3.0 now gets us the Gartner Hype Cycle with the Key Characteristics:

Location-aware analysis
Person-centred analysis
Context-relevant analysis
Mobile visualization & HCI

This is where we see the jump when we relate to places like Palantir that is now in the weeds prepping for war. Tech Crunch (at https://techcrunch.com/2016/06/24/why-a-palantir-ipo-might-not-be-far-off/) mentioned in June that it had taken certain steps and had been preparing for an IPO. I cannot say how deep that part was, yet when we line up a few parts we see an incomplete story. The headline in July was: ‘Palantir sues investor Marc Abramowitz for allegedly stealing company secrets‘, I think the story goes a little further than that. It is my personal belief that Palantir has figured something out. That part was seen 3 days ago (at http://www.defensenews.com/articles/dcgs-commentary), the two quotes that matter are “The Army’s Distributed Common Ground System (DCGS) is proof of this fact. For the better part of the last decade, the Army has struggled to build DCGS from the ground up as the primary intelligence tool for soldiers on the battlefield. As an overarching enterprise, DCGS is a legitimate and worthwhile endeavour, intended to compute and store massive amounts of data and deliver information in real time“, which gives us (actually just you the reader) the background, whilst “What the Army has created, although well-intentioned, is a sluggish system that is difficult to use, layered with complications and unable to sustain the constant demands of intelligence analysts and soldiers in combat. The cost to taxpayers has been approximated at $4 billion“, gives us the realistic scope and that all links back to the Intelligence Community. I think that someone at Palantir has worked out a few complications making their product the one winning solution. When I started to look into the matter, some parts did not make sense, even if we take the third statement (which I was already aware of long before this year “In legal testimony, an Army official acknowledged giving a reporter a “negative” and “not scientific” document about Palantir’s capabilities that was written by a staff member but formatted to appear like a report from the International Security Assistance Force. That same official stated that the document was not based on scientific data“, it would not have added up. What does add up (remember, the next part is speculative), the data links required in the beginning of the article, have to a larger extent been resolved by the Palantir engineers. In its foundation, what the journal refers to as BI&A 3.0 has been resolved by Palantir (top some extent). If true, we will get a massive market shift. To make a comparison, Google Analytics might be regarded as MSDOS and this new solution makes Palantir the new SE-Linux edition, the difference on this element could be that big. The difference would be that great. And I can tell you that Google Analytics is big. Palantir got the puzzle piece making its value go up with billions. They could raise their value from 20 billion to 60-80 billion, because IBM has never worked out that part of analytics (whatever they claim to have is utterly inferior) and Google does have a mobile analytics part, but limited merely as it is for a very different market. There have always been issues with the DCGS-A system (apart from it being as cumbersome as a 1990 SAS mainframe edition), so it seems to me that Palantir could not make the deeper jump into government contracts until it got the proper references and showing it was intentionally kept out of the loop is also evidence that could help. That part was recently confirmed by US Defense News.

In addition there is the acceptance of Palantir Gotham, which offered 30% more work with the same staff levels and Palantir apparantly delivered, which is a massive point that the Intelligence groups are dealing with, the lack of resources. The job has allowed NY City to crack down on illegal AirBnB rentals. A task that requires to connect multiple systems and data that was never designed to link together. This now gets us to the part that matters, the implication is that the Gotham Core would allow for dealing with the Digital data groups like Tablet, mobile and streaming data from internet sites.

When we combine the information (still making it highly speculative) the fact that one Congressman crossed the bridge (Duncan Hunter R-CA), many could follow. That part matters as Palantir can only grow the solution if it is seen as the serious solution within the US government. The alleged false statements the army made (as seen in Defence News at http://www.defensenews.com/articles/dcgs-commentary) with I personally believe was done to keep in the shadows that DCGS-A was not the big success some claimed it to be, will impact it all.

And this now links to the mentions I made with the Academic paper when we look at page 1174, regarding the Emerging Research for Mobile Analytics. The options:

Mobile Pervasive Apps
Mobile Sensing Apps
Mobile Social Networking
Mobile Visualization/HCI
Personalization and Behavioural Modelling

Parts that are a given, and the big players have some sort of top line reporting, but if I am correct and it is indeed the case that Palantir has figured a few things out, they are now sitting on the mother lode, because there is currently nothing that can do any of it anywhere close to real-time. Should this be true, Palantir would end being the only player in town in that field, an advantage corporations haven’t had to this extent since the late 80’s. The approach SPSS used to have before they decided to cater to the smallest iteration of ‘acceptable’ and now as IBM Statistics, they really haven’t moved forward that much.

Now let’s face it, these are all consumer solutions, yet Palantir has a finance option which is now interesting as Intelligence Online reported a little over a week ago: “The joint venture between Palantir and Credit Suisse has hired a number of former interception and financial intelligence officials“, meaning that the financial intelligence industry is getting its own hunters to deal with, if any of those greedy jackals have been getting there deals via their iPhone, they will be lighting up like a Christmas tree on those data sets. So in 2017, the finance/business section of newspapers should be fun to watch!

The fact that those other players are now getting a new threat with actual working solutions should hurt plenty too, especially in the lost revenue section of their spreadsheet.

In final part, why did I make the No Man’s Sky reference? You see, that is part of it all. As stated earlier, it used a formula to create a planet sized planet. Which is one side of the equation. Yet, the algorithm could be reversed. There is nothing stopping the makers to scan a map and get us a formula that creates that map. For the gaming industry it would be forth a fortune. However, that application could go a lot further. What if the Geospatial Data is not a fictive map, but an actual one? What if one of the trees are not trees but mobile users and the other type of trees are networking nodes? It would be the first move of setting Geospatial Data in a framework of personalised behavioural modelling against a predictive framework. Now, there is no way that we know where the person would go, yet this would be a massive first step in answering ‘who not to look for‘ and ‘where not to look‘, diminishing a resource drain to say the least.

It would be a game changer for non-gamers!

special_issue_business_intelligence_rese

Facts, Fiction or Fantasy

It is the elementary consideration of the three F’s, when we look at the information in regards to the Ukraine. It is not whether we give value or credibility of the news we see coming from Sky News, the Dutch NOS, BBC, CNN or even Fox News. There is a side that remains largely unspoken by many of them.

We see the news on how it is written on how these poor, poor Ukrainians are getting pummelled back into the anti-freedom group called ‘the Russian Federation’. Is that actually a truth?

Yes, we all notice on how well organised and well-armed these pro-Russian antagonists are, but are we seeing all the information correctly? Consider that not a few or a dozen people are in favour of these so called referendums, no; the people are out and about in hundreds and thousands. Many are singing their ‘old’ Russian songs and anthems. This is at the heart of the missing information. Consider that we see a lot more US involvement, whilst Kiev is now asking for the ‘Blue Helmets’ (UNIFIL) intervention. These people are about to get more support in 2 weeks, then the entire Syrian nation got in three years. I hope you remember that little escapade. It is still going on and the amount of casualties remain rising in Syria.

So, why are we all up in arms about Ukraine? Is it because some in Kiev want the European values and we are so upset about those who do not want to share ‘our’ way of life? Consider that the news has all been about implying that these acts are all orchestrated by the Kremlin and whilst it sounds really fun to hear about some politician who is about to get his assets frozen, nothing real can be done. By the way, can anyone tell me when the American Politicians or Wall street big bosses got their assets frozen?

The Ukrainian mess is blowing out of proportions in two ways. The first was the start of the Crimea and in specific the way the west and others responded to the events. I will always consider the fact that Russia did have some involvement here to some extent. The reason is that not having their fingers on the pulse whilst there is a massive naval base there is just not an option. They might not have intervened, or they remain silent on actions, but they knew what was going on. It was in their interest to pretend to be the non-observant here. Yet, that story does not reflect on the other parts of the Ukraine. A simple look at the map can tell us that. The Crimea was a military power point; the rest of the Ukraine is not. It is so simple for Russia to stand at a distance as see this all go up in flames and then offer ‘humanitarian’ aid.

The part that western news is ignoring is the shouting of the people that they have had enough of Kiev corruption. In their mind this will only lead to even worse times. Can we even blame them? Look at what the IMF has wrought (not through their actions through), Greece, Italy, Ireland, Spain and Cyprus. Massive debts, then IMF/EU financial support and after that austerity and continues after it started to choke a population. Government administrations get re-elected, no one goes to jail and some end up with a massive amount of money and favours. Is it such a leap of faith that Ukraine, a nation where corruption is such an issue, a place where now its population is just too scared to see what happens next? Consider the news in the last week, where we read that Christine Lagarde stated that the IMF was no longer forcing structural changes (http://www.sbs.com.au/news/article/2014/04/13/imf-no-longer-forces-structural-change). Was that just a small illumination of change as fear is gripping certain population groups? Consider the statement that was given last week that ‘the IMF was a victim of US politics‘, it is enough to scare many people. The statements of the IMF, which were also stated by Australian Treasurer Joe Hockey, that the US seems to be playing their own political games on regards to the IMF. None of these issues were raised, even though it is stated in several sources that the Ukraine is about to receive 9 billion in aid from the IMF. Now, I am not objecting in regards to the aid, yet, whilst it is known by all the players above a certain levels (at least 4 levels below Lagarde, Obama and Putin), that the Ukraine has a history and environment of corruption. None of that is properly addressed, so whilst 9 billion will go to the Ukraine, how much will end up out of the hands of the corrupt? Misreading gas meters, government invoices and the list goes on, how much of those will get paid by the 9 billion? Still wondering why the Ukrainian people are so anxious?

None of these matters are looked at (with proper levels of investigation) by the press, which makes for some of these newscasts a negotiable level of ‘pro-western’ advertisements, making the situation worse.

What the press is unwilling to illuminate, is that at the centre of these troubles are the pro-western politicians. They had no issue disposing of its former president, yet when they themselves are rejected by the Crimea and as it seems by the people at large, everyone shouts foul!

That part is an issue, no matter how many journalists ignore it. It is of course also a nice point of light as well; my income might drastically improve if the cold war is back. There is of course the badge of benefits we see with new movies (like a new impossible mission going up against their old adversary), the video games and in my case more data analyses. All those international locations that would need Palantir Government installed, trained and consulted upon.

Is this the reality? I do not know, the pressure between east and west is growing, so it remains a consideration. Consider however the events in Syria and that red line that was drawn (by the US), nothing happened. Is it because US intervention might get some of their oil benefits revoked? Is Syria not an interesting nation? (Which seems odd, as the pressures there would influence their long-time ally Israel.) So what is the press not investigating and what are we not getting told in this instance?

Consider that when you watch the news tonight and listen to what they say exactly, because you will hear suppositions and carefully phrased implied events, but where were the facts and more important, why are we not getting all the facts? That last one is important, as it turns a fact driven newscast into a work of fiction or even fantasy, which is getting the Ukrainians so angry and bothered.

In the end I still ask the question that is at the centre of this all. Why did the EEC not let the Ukraine be? This is not a statements against dealing with the Ukraine as a business partner, but in the light where the economies are down to such a degree, when the EEC is still dealing with the new partners and the overall debt levels are far exceeding acceptable levels in many of the EEC nations, growing is not a solution, it is a sure path to implosion, which will leave most of the EEC in a destitute state. That part is also seen as the two big national influencers, namely the French ‘Front Nationale’ and the British UKIP. When they do get the referendum to fall in their favour, the EEC will be in a mess that they will not be able to fix. Is the adding of as many nations as possible a desperate act to float the EEC at that point? (That was an actual question I am phrasing myself!)

The last one is likely to be a mere speculation (read fiction), from my side. Yet, considering the steps as we saw the EEC change and grow from 2008 onwards, after economic blow after blow. Now Greece is selling bonds again, whilst at present, their economy is in no way ready to deal with the old debts as well as the additional new ones. Are you still surprised to see the Ukrainian actions?

I am not stating that Russia is in such a great state, but there is every indication that they are not in a bad state either (with massive parts if Europe depending on Russian Gas), add to that, the fact that the Middle East is now diversifying by making Russian arms deals and other deals, which should indicate that they will order less from the west. Cars, electronics and other needs are now more and more moved to Asian makers like China, India, Myanmar et al. Some was already there, but slowly the list of migration is growing. Australia will lose massive amounts of jobs as the car industry moves away (not one brand, but all brands within the next 36 months). We see that airlines are slimming down and as the news reaches us day after day, often just after some ‘good’ news reached us, the balance is not looking good. The west is becoming less and less the place to be.

I do agree that the economy is slowly getting better, but it is also changing. Both have an impact on most of us and I still believe that actual economic improvements are not enjoyed by many of us until late 2015. All these factors are linked, as they are told to all. This is because the Ukrainian people are also watching the news, reading it on the internet and the picture shown is not a good one. So, when they felt that they were about to get the short end of the stick, they all rose up, because the devil you know (Russia) beats the devil you don’t (EEC). That part the big bosses all forgot about and when they applied pressure, they lost the Ukraine. Now the escalations there might not be so much orchestrated, but the stories, as they came from their ‘new’ government is sounding less and less honest in their ears. They want the old days back and in all fairness, can we blame them? Moreover, are the involved nations even happy to add another nation who is on the brink of bankruptcy?

These questions have not been dealt with at all. The last one is one we should all ask ourselves. Why intervene in the Ukraine, whilst politicians have no solution at all for those in hardship and dying in Syria? That issue reflects directly on the people of Jordan and Palestine, especially after a second chemical attack, whether we believe these events to be stories of fact, fiction or fantasy. We are witnessing iterations of ‘the cost of doing business’ on a global scale. It is however the local people who pay the bill through taxation and the Ukrainians seem to be very unhappy about the changes and the bill they will get presented with.

For free or for naught?

It is less than a day after I wrote the previous blog ‘The danger ahead’, now I read in the Guardian (at http://www.theguardian.com/world/2014/jan/27/nsa-gchq-smartphone-app-angry-birds-personal-data) that the quote I made in yesterday’s blog “Speed and disregard of proper development has allowed for open access to many computers and devices, which allows for almost complete collection and stored and such storage can only be done by just a few. This open level of availability allows the NSA and GCHQ (amongst others) to collect open source intelligence, hoping to gain the upper hand in the war on terror.“, which is close to what the Guardian reported, as well as what is currently shown on Sky News!

At this point, I am looking at a few issues and the more I look at the data that the press is stating, the more I see that Edward Snowden is more than just a traitor. He claims being a victim in a German TV interview (at http://www.dw.de/wanted-dead-by-us-officials-snowden-tells-german-tv/a-17388431), where he speaks the fear that he is being targeted for long term sleep therapy (aka ‘terminal sleep’).

The ‘problem’ is that the issue is not just Snowden. The more I look into the breaches, the more I look into a possible functional approach on the way the NSA server parks (plural) are set up, the more I am convinced that not only was Edward Snowden not alone in this all, I feel some level of certainty that this person might still be in the NSA, endangering both NSA and GCHQ as well as other allied monitoring agencies.

The humongous amount of ‘revelations’ that are claimed in the name of Snowden do two things. First of all it turns Benedict Arnold in a stumbling saint (I just had to wash my mouth with soap for making such a claim). Linked to this is the fact that the many dozens of operations as his ‘revelations’ seem to touch on would have been on at least a dozen of servers (as projects are spread around). The fact that NSA uses an upgraded edition of SE-LINUX means that a system with logs and mandatory access control cannot get transferred to such a degree. The fact that IT and security monitors it all, as well that he was civilian contractor means that his name should have popped up a dozen times. Even if he used other accounts, the logs should have triggered alerts all over the field when they were scanned through solutions not unlike a program like Palantir Government.

The claims I am making are growing in reliability with every ‘revelation’ that is being made. There is however another side that is now the consequence of all these whingers and whiners about ‘their privacy‘ (at http://www.theguardian.com/world/2014/jan/27/tech-giants-white-house-deal-surveillance-customer-data). We now enter a field where it is important to realise that the new situation could be regarded as a danger.

It is linked to a previous newscast where President Obama was considering moving telephony data out of government hands (at http://www.washingtonpost.com/blogs/the-switch/wp/2014/01/23/government-privacy-board-members-say-shifting-nsa-data-to-third-parties-is-a-bad-idea/)

As stated before, this is a really bad idea. Consider that criminals, if enough money is in play, can use places like HSBC to launder their money (I am not talking about forgetting your wallet whilst washing your jeans), but the idea that commercial enterprises can get away with these events for just a 5 week fee (at http://www.forbes.com/sites/afontevecchia/2012/07/16/hsbc-helped-terrorists-iran-mexican-drug-cartels-launder-money-senate-report-says/, as well as http://uk.reuters.com/article/2014/01/23/uk-standardbank-fine-idUKBREA0M0LF20140123) is a lot more dangerous than many realise. Handing data storage out of government hands is just too dangerous. I am steering away from the issue whether the monitoring program should go on or stop. The intelligence community needs to do what it needs to do. Leaving that data with third parties is just not an option. The worst case scenario would see the US government paying out billions if any data leading to a registered IP ends up in ‘other’ hands. Once that evidence is ever given, the US would lose whatever credibility they ever thought they had.

At this point the title can be used as a joke. What is the difference between for free and for naught? Someone got rich for free, the US got rich for naught! That would end up being the reality of a project that was meant to map levels of global terrorism. This joke only gets stronger when we see another ‘view of shock’, but now from Google CLO David Drummond (at http://www.bbc.co.uk/news/world-25911266). It is hard to state against his view, or the premise of the company. These carefully pronounced statements from legal eagles are to be expected from many firms for some time to come. There is however a commercial positive view (at http://www.bbc.co.uk/news/technology-25914731). Here we see how entrepreneurs in makeup and clothing are showing options to avoid detection. In more than one instance it is stated to be metal based, so standing next to airport detectors should be fun soon enough. I wonder how much more would get checked when the boxers or briefs are also metal based.

So whether we get entertainment for free or fashion for naught will be discussed by many soon enough, the main fact remains. If we want to remain safe, then data needs to be collected. It is not for free, or for naught. It is for the simple reason that the world is filled with bad people; some will go any distance to hurt as many as they can. Our governments have a duty to keep us safe, it is only fair that they are given the tools, the methods and the opportunity to do so.

This does get us to the final part (or final side) to these events. This morning, the Guardian (at http://www.theguardian.com/world/2014/jan/28/microsoft-rules-out-back-door-access-to-mps-electronic-communications) reported on backdoor access allegations. The quote “Both Ludlam and South Australian independent senator Nick Xenophon have been concerned about the security of Australian parliamentary communications since the Prism surveillance program was first revealed by National Security Agency contractor-turned-whistleblower Edward Snowden.” gives the information that was the part of all this. So again we see more resources squandered in regards to Snowden. Do not get me wrong, the question by both Ludlam and Xenophon is fair enough and as such it should be looked at. Whoever wants access to certain information, which might always be the case, could consider Intruding a system, which, unless you are a real expert is getting harder and harder, as it should be.

Yet, capturing and copying frames sent over a router system makes a lot more sense. You just capture it all and decrypt it later. Now, most people will not have the ability to do this, but consider the amount of elements to get this all from user1 to user2 via server X. If you think that this is highly encrypted hard to achieve effort, then think again. The more common the method used, the easier it is to read into it. So, there is a level of entertainment as we see leagues of technicians concentrate on the door of the bank vault, whilst in reality one of the walls is missing. To give you another example, we take a look at a paper by Daehyun Strobel, Benedikt Driessen, Timo Kasper et al (at https://eprint.iacr.org/2013/598.pdf). As we look at the quote “Despite the fact that nowadays strong and well-analyzed cryptographic primitives are available for a large variety of applications, very weak cryptographic algorithms are still widely deployed in real products all over the world.” This relates to the IT issue as, we might have secure servers and powerful password rules, but files are send from one computer to another via the ‘internet’, which goes via a router system (no matter how you twist or turn it). So, as someone gets to any router on the track and wireshark’s the traffic, the stream can be rebuilt. From there the hacker still faces a few obstacles, but you better believe that above a certain skill level, this data can be retrieved. So what exactly are we all crying about?

Tag Archives: Palantir Government

Room for Requirement

Facts, Fiction or Fantasy

For free or for naught?

Recent Posts

Archives