Tag: software development

  • Investing In Requirements

    Investing In Requirements

    Custom software can be very expensive and scary, but investing in requirements can help you improve the likelihood of success. However, we often want to see things being built rather than simply think about them. That creates two things that fight against each other (get it done or think it through) in many projects. Thus, we must consider early work on requirements and design as an investment. We pay for it in patience, but it saves us in almost every way down the road.

    Investing In Requirements Is An Easy Choice

    There is so much value in thorough requirements gathering that it seems impossible for anyone to pass over this crucial step. Unfortunately, it is a common problem and one of the reasons many software projects fail. Yes, we want to see “work getting done,” but the only reason that is a problem is because we are not yet seeing the end product. I like to use the analogy of going on a road trip. You are not getting closer to the destination until you start driving. However, if you do not know where you are going or the obstacles on the way, it can be extremely costly. Those that use modern GPS maps or apps like Wayz often are redirected around problem areas and potentially save hours of drive time. That is the same with software development. We are not writing code but investing in requirements, so we have a better idea of the path ahead. That allows for proper estimates and planning.

    We Can Adjust As We go

    One argument we fall prey to is that we can start building the software and easily adjust as we go. While there is truth in that, and we do not need to know the whole journey in detail, we do need to plan for each stretch. That is what the Agile approach embraces. It takes a bit from both worlds and gets us on the road faster while still ensuring we are planning enough to avoid major issues. When we use the road trip analogy, it is like planning the trip but then being open to adjusting due to construction or traffic delays. However, the key is that we need to know at least some of our path before we start. In a software project, this can be done via high-level requirements and core feature definition. Be warned that is a possible approach, but it can still suffer from wrong turns and proceeding down dead ends.

    Back On The Main Road

    The discussion of Agile and its strengths and weaknesses is for another article. Let us return to a focus on requirements that applies to any software project. Modern software systems are highly complex. There are tools that can help us to create or generate thousands of lines of code quickly. However, that speed can be just a faster way to create a big mess. I have seen many projects stumble and fall due to this issue. The problem may seem simple, but the details are highly complex.

    For example, let’s consider the simplest of problems to solve: keeping contact information for your customers so you can serve them. How hard can it be? There is a name and address and maybe a phone number or email. Oh wait, what if the customer needs a title, or is a company with multiple contacts? What if the billing contact is different from the key decision maker? Do we care if they have bought from us before, or do we track leads? Should we keep notes each time we talk to them? The list goes on and on. A simple problem can quickly become a complex maze of decisions. These can be very costly to make if we have already made a lot of progress on our software journey. Changes can require updates to code the database, the user interface, and even the partners or libraries utilized. Backing up is far more than that BEEP, BEEP, BEEP a large vehicle makes.

    How Do We Get The Requirements We Need?

    Custom software development is just a way to solve problems on a big scale. Thus, we can use basic problem-solving techniques to guide us in creating a list of requirements that we can be confident in. Here is a short list of questions that you can use to take your requirements definition up to another level.

    • What does this need to do?
    • What does success look like?
    • Who needs to know about task completion?
    • Where can it fail?
    • Who needs to know about a failure, or how do they need to be told?
    • Is there anything else it needs to do?
    • Can any user do it?
    • Does every user experience it the same way?
    • What did we forget?
    • No, really, what else did we forget is a part of this process. Walk through each step as thoroughly as possible.

    The above question may not seem like much. However, they are easy to overlook. I have worked on dozens (probably hundreds) of software projects and still need to step back and review these questions each time. We can easily get lost down a rabbit trail and skip a step. A thorough review before building out requirements is a huge step in the right direction.

    Play It Back To Me

    A good way to ensure clear communication is to have someone repeat back to you what you said in their words. Do the same thing with your requirements. Walk through your problem and the proposed solution using only what is listed in the requirements. This action often brings up more requirements and details than even the last two questions listed above.

    Next Steps

    Feel free to schedule a time to discuss your next project with us. We are happy to help you with investing in requirements and improving the overall success rate of software projects. Our experience has taught us a lot about the pitfalls and challenges of custom software. Likewise, we have an e-book that can help you explore all the steps in building software, including a few templates. However, we ask that you share an e-mail address so we can send you a copy. We will add you to our monthly newsletter, but you can unsubscribe anytime. Your data is not shared with anyone else. Learn more about our book here.

  • Data Scraping and System Integration

    Data Scraping and System Integration

    The propensity of data scraping and system integration projects shows we are all figuring out the value of data in our solutions. However, there seems to be a disconnect between this common pursuit and the paths and challenges it offers. The ideas of data scraping and system integration are often seen as very different, but they are the same. We can scrape data from a site or application to integrate with it. Yes, that is typically a one-way integration, but it can go both ways.

    Methods Of Data Scraping And System Integration

    This topic is best started with a list of how to move data from one application to another. Data scraping is one way and likely the most complicated and fragile approach. Therefore, it is essential to consider other options before tackling that “worst-case” option. Integration can be achieved through the following techniques. It is not a comprehensive list but a good overview. For example, we will skip the manual option, which is always available.

    • Web Scraping
    • Direct Link/SSO
    • Application Programming Interface (API)
    • Data feeds (RSS or other)
    • FTP/SFTP and data import/export

    Each option has pros and cons worth examining before you start your next integration project.

    Web Scraping

    This approach is as close to manual integration as you can get. The way it works is the application has a process that connects to the desired system in a way that is identical to a user. For example, the application will open a browser window, enter a URL, send login information, click on buttons, and then copy data from screens and paste it into its data stores. You can mimic a data scraping process by logging into your chosen site, navigating to a desired page, and then writing down values you see on the page. This is a brute-force approach that requires nothing from the integrated system. However, some sites and applications restrict usage and timing. That can frustrate scraping attempts and sometimes is intentional.

    Direct Link/SSO

    This approach is often preferred over scraping, although not much different. We scrape data to pull it from another application into our own. A direct link or SSO configuration can bring the application into our own. The user often sees this as a window within the primary application or as popping open a new window or tab. It can sometimes be a very fluid user experience that is easy to implement. While that integrated application binds the user to its experience, it also will be familiar to them if they have a history of using it separate from your solution.

    Application Programming Interface

    The API solution often is the smoothest way to integrate data between systems. It can be complex and also comprehensive. There are many ways to achieve this, but they can be simplified down to two applications “talking” to each other. While it requires the interface to be defined and stable, it is one of the best ways to create a solution written once and requires little maintenance.

    Data Feeds

    A data feed is similar to an API in many ways. The main difference is that an API can be interactive and done at a data item level. A data feed is a data stream or similar query that can then be processed and pulled into a system. That can be beneficial when real-time integration is not needed. It is often associated with “nightly processing” and can remove the burden of extra processing during peak times of the day.

    Data Import/Export

    A step down from a data feed is an import/export. This can be done on an ad hoc basis and is the equivalent of taking a snapshot of a data block. Data feeds provide a data stream, while an import and export add an integration (import or export the data) and work with files. There are many similarities, but the import/export process does tend to be more static and limited in its use.

    Next Steps

    Feel free to schedule a time to discuss your next integration project with us. Our experience has taught us a lot about the pitfalls and challenges of these project types. Likewise, we have an e-book that can help you explore all the steps in building software, including a few templates. However, we ask that you share an e-mail address so we can send you a copy. We will add you to our monthly newsletter, but you can unsubscribe anytime. Your data is not shared with anyone else. Learn more about our book here.

  • Solving Business Problems, Not An Experiment With The Latest Tech

    Solving Business Problems, Not An Experiment With The Latest Tech

    The modern landscape makes solving business problems an exercise in finding the right provider most of the time. In my experience, software consultants and solutions providers come in three types. Some are heavily invested in existing technology and want to use that for every customer. Then, some want to jump on every new technology and use clients as guinea pigs. The third type focuses on the client and the best solution. Once that solution is defined, they decide how to implement it. These are the ones you want for your project. The challenge is figuring out how to weed through the options.

    Solving Business Problems Is The Focus

    I would say start with the website or proposal. However, you might come across a provider that does not have a website and has not sent you a meaningful proposal. Those are easy to ignore. It is hard to take a technology business seriously that either does not have a website or that has one that looks like it was built by a child a few decades ago. Here are some things a good provider will have on their website. The more of these, the better, and the more details, the better.

    • They talk about successful projects and solutions rather than technology stacks.
    • There are multiple clients represented, whether across projects or stories.
    • The focus is on problems they can solve as opposed to buzzwords.
    • Your non-technical friends, family, and coworkers can understand what they do without searching for the meaning of several terms.
    • There is a start and finish to the projects they highlight.

    While all of the above items are not required, a complete lack of this sort of content should raise warning flags. The alternatives you might see will list a bunch of technology they use (the stack) or provide links to cutting-edge vendors. They likely have the wrong focus when the site looks like an article from a tech news site or program.

    Finding That Perfect Match

    Once you have whittled down your list of prospective providers to those that actually solve problems, your next step is finding a match. The best providers will talk about solving problems you have faced or need to solve. For example, a vendor that spends a lot of time discussing how to market your business may not be the best if you need to track sales or integrate systems. On the other hand, a vendor that talks about bringing customers to your website may be the perfect fit.

    Be Prepared

    This may make you wonder how to match your problem to their skills. That is a good question and does require you to start by ensuring you have correctly understood and defined it. That is a mistake many companies make. They look for a provider and expect the provider to help define the problem and then provide a solution. While that is an acceptable approach, it will likely be more expensive. You will end up in a situation where you are in that old description of paying a consultant to use your watch to tell you what time it is. They are going to be an essential part of crafting a solution. However, you will want to invest the time in defining your problem to solve and related requirements as much as possible.

    A software project can be an expensive process. Think of it like a house or a mansion. You would not go into one of those projects blind or without firm ideas of what to expect. Do the same with your business. Take the time to examine your processes and the pain points you want to reduce or resolve.

    Next Steps

    We have an e-book that can help you explore all the steps in building software, including a few templates. However, we ask that you share an e-mail address so we can send you a copy. We will add you to our monthly newsletter, but you can unsubscribe anytime. Your data is not shared with anyone else. Learn more about our book here.