Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US20210064053A1) and Cella (US20190324444A1).
Regarding claim 1, Kim teaches an information display device system, comprising: a display device configured to display information (Kim; ¶0089-92, describes an output unit that generates visual/audio output and includes a display module. This teaches a display device configured to display information.), a control device configured to control the display device, (¶0101-105, describes that processor 270 controls operation of the user interface apparatus and that, when the processor is not included, the user interface apparatus operates according to control of another process within the vehicle or controller. This teaches a display control device to control the display device.), and a recording device configured to record transport vehicle travel data and path map data, (¶0500-503, describes the processor transmitting vehicle-status data to the Event Data Recorder (EDR) for storage. Kim also describes, ¶0249-255 and ¶0284-296, storing/processing HD map data and generating horizon map/path data. This teaches recording travel data and path map data.), the transport vehicle travel data being data comprising, in an associated manner, position information indicating a position of a transport vehicle (Kim; ¶0500, describes EDR data includes vehicle status information such as speed/brake/acceleration/RPM/steering angle, and, ¶0502, includes current position information and time information associated with the data. This teaches position information associated with vehicle status data using time/metadata.), the path map data being data of a map of a travel path on which the transport vehicle has traveled (Kim; ¶0249-255 and ¶0290-296, describes HD map/horizon map data and horizon path data (trajectory) generated/maintained by the processor. This reads on map/path data for a travel path used by the vehicle.), the control device configured to perform a map display process of displaying, at least part of the map based on the path map data (Kim; ¶0443, describes displaying lane guidance/lanes on the map and surrounding information on a vehicle display. This teaches displaying at least part of the map based on path map data.), a transport vehicle display process of displaying a transport vehicle mark indicating a position of the transport vehicle at a target display time point on the map displayed through the map display process, (Kim; ¶0444, describes a graphic object indicating the location of the vehicle may be included on the map/lane depiction and, ¶0502, enabling selecting a target display time point for the displayed position. This teaches displaying a vehicle mark indicating a position of the vehicle at a target display time point on the map.)
However, Kim fails to teach, but Cella teaches vibration information indicating vibration measured in the transport vehicle, (Cella describes (¶0023) sensor streams that are continuous vibration data, and (¶0431-432) acquiring/recording and display of streamed sensor waveform data. This teaches vibration information measured in the system.), the control device configured to define a first display area and a second display area in a display screen of the display device, (Cella describes (Fig. 26 and ¶0432) a single screen divided into multiple windows (four windows showing four channels). This teaches first and second display areas within a display screen.), a graph display process of displaying a vibration result graph showing the vibration information at least at and around the target display time point on the second display area (Cella describes (¶0431-432) displaying a data stream and allowing selection of a contiguous subset of the stream and displaying the stream data in waveform windows. Because the waveform window displays a time interval (or a selected subset), it displays vibration information at and around any chosen/target time point within that interval.), a position mark display process of displaying a position mark indicating a position on the vibration result graph corresponding to the target display time point on the second display area (Cella describes (¶0431) navigation to a particular time point and (¶0434) full cursor control on the displayed waveform/spectrum. Also shown in Fig. 28, the cursor placed at the selected time point is a position mark indicating a position on the vibration result graph corresponding to the target display time point.), and associating vibration information with position/time (Kim teaches, ¶0502, that the recorded tracel data includes current position information and time information. Cella further describes, ¶0359, map/visualization coordinates can include real-world location coordinates (geo-location) and time-based coordinates for representing sensor signals in map-based visualization. This reads on recording/using vibration data in association with position/time when combined with the EDR logging framework of Kim). It would have been obvious to one or ordinary skill in the art, before the effective filing date, to modify Kim’s vehicle EDR and map display system to incorporate Cella’s vibration sensing and multi-window waveform visualization with timepoint cursor controls by displaying Kim’s map on a first display area and Cella’s vibration result graph on a second display area using Cella’s multi-window approach (Fig. 26 and ¶0432) with the benefit of enabling analysis/correlation between the vehicle’s position on the map with the vibration waveform at a selected time.
Regarding claim 2, Kim in view of Cella teaches the information display system according to claim 1, wherein: the transport vehicle travel data further comprises vehicle state information indicating a state of the transport vehicle, and the vehicle state information is associated with the position information and the vibration information (Kim; ¶0500, describes that the data provided to the EDR may include data related to operations of vehicle components such as speed, brake information, acceleration, engine rotation speed (RPM), and steering angle. Data indicating speed, brake status, acceleration, RPM, and steering angle reads on vehicle state information indicating a state of the transport vehicle. Kim further describes, ¶0502, that the data provided to the EDR includes current position information related to the vehicle and time information at the time that the data has been transmitted. Cella discloses, ¶0334, that data from multiple sensors is multiplexed for storage of a fused data stream in a time series so that the values remain linked in time and, ¶0308, that operation parameters such as RPMs may be collected on data channels in parallel with vibration waveforms and applied to time marks. Together, vehicle state information and vibration information are both time-aligned with position/time metadata, associating vehicle state information with both position information and vibration information. This teaches that vehicle state information is associated with the position information and the vibration information.) in the graph display process, the control device displays, together with the vibration result graph, a vehicle state graph showing the vehicle state information at least at and around the target display time point on the second display area (As previously discussed in claim 1, Cella teaches, ¶0432, displaying multiple sensor data streams as waveform displays in multiple windows on a screen. Cella describes, Fig. 26 and ¶0432, that the screen may include four windows that show stream data from four channels simultaneously. Cella further describes that operational parameters can be collected on data channels in parallel with waveform data such as vibration and that these preset markers (operation parameters) can be fed into the data acquisition system directly and collected in parallel with waveform data. Cella describes, ¶0308, that markers can be applied to time marks within the raw waveform data. In the combination, Kim’s vehicle state information can be displayed as additional graphs (time-series graphs) alongside the vibration result graph using Cella’s multi-window/multi-channel display approach with parallel data collection. Cella’s teaching, ¶0308, of collecting operation parameters with vibration waveforms and, ¶0432, displaying multiple data streams simultaneously in separate windows reads on displaying a vehicle state graph together with the vibration result graph, where both show data at and around target display time point, enabling correlation of multiple sensor parameters for system diagnosis. This teaches displaying, together with the vibration result graph, a vehicle state graph showing the vehicle state information at least at and around the target display time point on the second display area.), and in the position mark display process, the control device displays a position on the vehicle state graph corresponding to the target display time point on the second display area using the position mark (As previously discussed in claim 1, Cella, ¶0434, teaches full cursor control on displayed waveforms and teaches, Fig. 25 and ¶0431, “advance to time point” and “retreat to time point” controls. Cella further describes, ¶0531, that sensor data from a plurality of sensors may be captured concurrently in a coordinated manner, such as repeatedly sampling each of the sensors synchronously or with a known offset. It would have been obvious to one of ordinary skill in the art to display Kim’s vehicle state graph (showing information over time) with Cella’s vibration result graph on the same display using Cella’s multi-window technique to provide the benefit of enabling users to correlate vehicle conditions with vibration events at time points for improved diagnosis and analysis.)
Regarding claim 3, Kim in view of Cella teaches the information display system according to claim 1, wherein: the transport vehicle travel data further comprises vehicle state information indicating a state of the transport vehicle, and the vehicle state information is associated with the position information and the vibration information (As previously discussed, Kim describes, ¶0500, the data provided to the EDR may include data related to operations of vehicle components, such as speed, brake information, acceleration, engine rotation speed, and steering angle, which reads on vehicle state information indicating a state of the transport vehicle. Kim further describes, ¶0502, that the data provided to the EDR includes current position information related to the vehicle and time information. Cella describes, Cella; ¶0334, that data from multiple sensors is multiplexed for storage of a fused data stream in a time series so that the values remain linked in time and, ¶0308, that operational parameters such as RPMs may be collected on data channels in parallel with vibration waveforms and applied to time marks. In the combination, vehicle state information and vibration information are both time-aligned with position/time metadata (Kim ¶0502), associating vehicle state information with both position information and vibration information. This teaches that vehicle state information is associated with the position information and the vibration information.) the control device further performs a vehicle state display process of displaying a vehicle state mark indicating the vehicle state information on the map displayed through the map display process (Kim describes, ¶0433, that when autonomous driving visibility information is displayed on a display mounted on the vehicle, guide lines for guiding lanes that can be driven on a map and information within a predetermined range based on the vehicle may be displayed and, ¶0444, that a graphic object indicating the location of the vehicle may be displayed. Kim further describes, ¶0500, vehicle state information including speed, brake, acceleration, RPM, and steering angle. Cella discloses, ¶0359-360, displaying map-based indicators of sensor data, describing that heat maps can display collected data with a presentation of a map that includes indicators of levels of analog and digital sensor data such as rotation, vibration, heating or cooling, and pressure, where coordinates may include real world location coordinates on a map of an environment. Cella further describes, ¶0361, that visual elements may include icons and map elements for representation of analog sensor signals and that colors, shapes, and sizes of visual overlay elements may represent varying levels of input along relevant dimensions. Cella describes, ¶0314, monitoring heavy-duty industrial vehicles and off-road industrial vehicles and, ¶0326, describes applications including navigation and guidance systems and vehicle tracking. It would have been obvious to one of ordinary skill in the art to modify Kim’s map display (Kim; ¶0443) to include a vehicle state mark indicating Kim’s vehicle state information (¶0500) on the map because Cella teaches displaying state indicators on maps using visual elements including icons and map elements that represent sensor conditions at locations (Cella; ¶0359-361) in vehicle tracking and navigation contexts (¶0314, 0326) and this modification provides the predictable benefit of presenting contextual vehicle state information to the user at the relevant map location for improved situational awareness. This reads on displaying a vehicle state mark indicating the vehicle state information on the map) the vehicle state mark is displayed in a manner corresponding to the transport vehicle mark displayed through the transport vehicle display process (Kim describes, ¶0444, that a graphic object indicating the location of the vehicle may be included on the map and that, ¶0502, vehicle state information is associated with position information via time information. Cella teaches that map elements can visually represent sensor conditions, describing that, ¶0359, if a machine is overheating, the heat map may show the machine in bright red and that colors may represent varying levels of input. Cella further describes, ¶0361, that visual elements including icons and map elements may be modified in color, shape, size, and motion to represent varying sensor levels and that such visual overlay elements can be overlaid on maps and camera views.). It would have been obvious to one of ordinary skill in the art to display the vehicle state mark corresponding to the transport vehicle mark (Kim; ¶0444) because Kim’s vehicle state and position information are associated with time (¶0502), Cella teaches modifying icons and visual elements with colors, shapes, sizes, and motion to indicate sensor state and overlaying such elements on maps (¶0359, 0361) and displaying the state mark corresponding to the position mark provides the predictable benefit of immediately conveying the vehicle’s operational status at its current location for immediate assessment by the user.
Regarding claim 4, Kim in view of Cella teaches the information display system according to claim 2, wherein: the vehicle state information comprises at least one of a travel speed of the transport vehicle, an acceleration state of the transport vehicle, an operation state of a traveler included in the transport vehicle, or a detection state of a sensor included in the transport vehicle (Kim; ¶0500, describes that the data provided to the EDR may include speed and acceleration. This teaches that the vehicle state information comprises at least one of a travel speed of the transport vehicle and an acceleration state of the transport vehicle.)
Regarding claim 5, Kim in view of Cella teaches the information display system according to claim 1, wherein: the control device further performs a path shape display process of displaying path shape indication indicating a feature of a shape of the travel path on which the transport vehicle has traveled on the second display area (Kim; ¶0251, describes that ADAS data may include road slope data and road curvature data and, ¶0249, that horizon map data may include ADAS data. Kim further describes, ¶0502, recording current position information and time information. The curvature and slope features from the map data (¶0251) can be determined at the recorded positions along the route actually traveled using the recorded position/time data (¶0502) to the corresponding path features in the map and displayed as a path shape indication for the traveled path the vehicle has traveled. Road curvature data and road slope data read on a feature of a shape of the travel path. Kim describes, ¶0443, displaying guide lines for guiding lanes that can be driven on a map and information within a predetermined range based on the vehicle, but does not explicitly disclose displaying the path shape indication on the second display area. Cella describes, Fig. 26 and ¶0432, a display screen with multiple windows showing stream data from multiple channels simultaneously. This teaches displaying path shape indication indicating a feature of a shape of the travel path on the second display area. It would have been obvious to one of ordinary skill in the art to display the path shape feature of Kim’s map data in an additional window on the second display area alongside the vibration result graph using Cella’s multi-window visualization approach (Cella; ¶0432) because both address displaying multiple sensor/path data streams for monitoring and analysis and the modification would provide the benefit of side-by-side correlation of path features and measured vibration for improved diagnostics.) and the path shape indication is displayed in a manner corresponding to a position at which the vibration shown in the vibration result graph is measured (Kim; ¶0502, describes that the recorded vehicle data includes current osition information and time information, which enables correlating measured sensor data (including vibration data, see claim 1) to a specific position along the travel path. Cella describes, ¶0431, that the data stream may be replayed with controls including advance-to-time-point and retreat-to-time-point and, ¶0434, full cursor control on displayed waveform data. Using Cella’s cursor and timepoint control (¶0431, 0434), the same selected timepoint (and corresponding position via Kim’s position/time association, Kim; ¶0502) is indicated on both the vibration waveform and the path shape indication, which would display the path shape indication in a manner corresponding to the position at which the vibration is measured. This teaches displaying the path shape indication in a manner corresponding to a position at which the vibration shown in the vibration result graph is measured).
Regarding claim 6, Kim in view of Cella teaches the information display system according to claim 1, wherein: the transport vehicle travel data further comprises detection area information indicating a state of a detection area for an object detection sensor included in the transport vehicle, and the detection area information is associated with the position information and the vibration information (Kim; ¶0502, describes recording vehicle travel data with time information and current position information in connection with data transmitted to the EDR and describes, ¶0500, recording vehicle status information such as speed, brake, acceleration, RPM, and steering angle. Kim describes, ¶0106, describes an object detecting apparatus for detecting objects located outside the vehicle (see also Figs. 5 and 6) and which, ¶0119, may include sensors such as a camera, radar, LiDAR, ultrasonic, and infrared sensors. Kim describes, ¶0443, displaying information within a predetermined range based on the vehicle. A predetermined range based on the vehicle reads on a detection area form an object detection sensor. Kim further describes, ¶0501, that data related to surrounding information sensed by a second sensor (safety sensor) may include information related to objects, road, and other vehicles located in the vicinity of the vehicle. The second sensor that detects objects reads on an object detection sensor included in the transport vehicle and the detected objects and surrounding information within the vicinity reads on detection area information indicating a state of the detection area. It would have been obvious to one of ordinary skill in the art to store the second sensor’s surrounding and object information in the same time and position-tagged travel record because Kim’s EDR already records vehicle status information and adding sensor detection data to the travel record provides the predictable benefit of enabling correlation of detected objects with vehicle position and conditions for improved event analysis.
Cella describes, ¶0334, that data from multiple sensors may be multiplexed into a fused data stream in a time series so that the values remain linked in time and describes, ¶0308, applying markers to time marks within data and collecting parameters in parallel with waveform data. In the combination, Kim’s detection area information (¶0443, 0501) is recorded as a parameter in the time series travel record so that the detection area information is associated with the recorded position information (Kim ¶0502) and the vibration information (see claim 1). This teaches the detection area information associated with the position information and the vibration information.) and the control device further performs a detection area display process of displaying a detection area mark indicating a state of the detection area on the map displayed through the map display process (Kim; ¶0443, describes displaying a map with information within a predetermined range based on the vehicle and describes, ¶0444, including a graphic object indicating the location of the vehicle on the map. Cella describes, ¶0359 and 0361, map based visualization with overlays and indicators of sensor data conditions and can use visual overlay elements such as colors shapes and sizes to represent varying levels of sensed data. It would have been obvious to one of ordinary skill in the art to display Kim’s detection area information as a detection area mark on the map using Cella’s map overlay and indicator techniques because both address presenting sensed conditions via visual indicators on map based displays and this modification provides the benefit of showing the information on the map for improved situational awareness. This teaches displaying a detection area mark indicating a state of the detection area on the map.) and the detection area mark is displayed in a manner corresponding to the transport vehicle mark displayed through the transport vehicle display process (Kim, ¶0444, describes displaying a vehicle location graphic object on the map. Cella, ¶0359 and 0361, teaches that map overlay elements can be configured to represent sensed conditions and displayed as visual elements tied to relevant real-world location representations. It would have been obvious to one of ordinary skill in the art to display the detection area mark corresponding to the transport vehicle mark by rendering the detection area mark anchored to/around/emanating from the vehicle mark with the benefit that the user can more quickly understand sensor coverage relative to the display position. This teaches the detection area mark displayed in a manner corresponding to the transport vehicle mark.)
Regarding claim 7, Kim in view of Cella teaches the information display system according to claim 1, wherein: the transport vehicle travel data further comprises travel speed information indicating a travel speed of the transport vehicle, and the travel speed information is associated with the position information and the vibration information (As previously discussed in claim 4, Kim, ¶0500, describes that the data provided to the EDR may include speed, which reads on travel speed information indicating a travel speed of the transport vehicle. As discussed previously in the rejection of claims 2 and 4, Kim, ¶0502, describes that the data provided to the EDR includes current position and time information, and Cella, ¶0334, teaches that data from multiple sensors is multiplexed for storage of a fused data stream in a time series so that the values remain linked in time and, ¶0308, that operational parameters may be collected on data channels in parallel with waveform data and applied to time marks. In the combination, travel speed information and vibration information align time and position/time metadata (Kim; ¶0502) which associates travel speed information with both position information and vibration information. This teaches the travel speed information is associated with the position information and the vibration information.) and the control device performs a replay display process of moving, in the transport vehicle display process, the transport vehicle mark relative to the map at a speed corresponding to the travel speed of the transport vehicle based on the transport vehicle travel data and moving, in the position mark display process, the position mark relative to the vibration result graph at a speed corresponding to a moving speed of the transport vehicle mark (Kim; ¶0444, describes that a graphic object indicating the location of the vehicle may be included on the map and, ¶0502 and ¶0500, that the recorded data includes current position information, time information and speed. Cella describes, ¶0431, that the data stream may be replayed with controls including advance-time-point, retreat-to-time-point, play, forward, fast forward, and other such controls and, ¶0434, describes full cursor control on displayed waveform data. It would have been obvious to one of ordinary skill in the art, before the effective filing date, to implement a replay display process that moves the transport vehicle mark (Kim; ¶0444) along the recorded path on the map during playback of the recorded travel data using Cella’s playback controls (¶0431) because both address time-indexed recorded data to users with the benefit of allowing the user to review and analyze the vehicle’s journey with time accuracy. During playback using Cella’s controls (¶0431), successive time-stamped positions (Kim ¶0502) are rendered, such that the vehicle mark’s movement corresponds to the recorded speed (Kim; ¶0500). Using Cella’s controls and cursor functionality (¶0434), the position mark on the vibration result graph moves through the waveform during playback and because both the map display and vibration graph moves corresponding to the moving speed of the vehicle mark, would synchronize the displays to show the vehicle’s position and corresponding vibration at each moment during replay. This teaches performing a replay display process of moving the vehicle mark relative to the map at a speed corresponding to the travel speed and moving the position mark relative to the vibration result graph at a speed corresponding to the moving speed of the vehicle mark.)
Regarding claim 8, Kim in view of Cella teaches the information display system according to claim 7, wherein: the control device further performs a vibration point mark display process of displaying, on the map, a vibration point mark indicating a point on the travel path at which a value of vibration indicated by the vibration information exceeds a predetermined determination threshold (Kim; ¶0443-0444, describes displaying a map with a graphic object indicating the location of the vehicle and, ¶0502, recording position and time information.
Cella describes, ¶0331, that measurement values including vibration may be tracked by a state machine and that where a measurement state exceeds a certain threshold, the system may track that state and produce output indicating an anticipated fault state. Cella teaches, ¶0310, that dynamic markers can correlate to data derived from waveform analysis, including operationally derived markers can correlate to data derived from waveform analysis, including operationally derived metrics such as alarm conditions and, 0361, describes highlighting unusual vibrations in visual displays. Cella further describes, ¶0359 and 0361, map-based visualization with indicators representing sensor conditions at locations on a map. It would have been obvious to one of ordinary skill in the art to analyze Kim’s recorded vibration information to identify points where vibration exceeds a predetermined threshold using Cella’s threshold-based state tracking (¶0331), techniques for deriving alarm conditions from waveform analysis (¶0310), and to display vibration point marks at the corresponding map positions (using Kim’s position/time association, ¶0502, and Cella’s map indicators, ¶0359, 0361) because both address monitoring sensor data for diagnostic purposes and alerting users to abnormal conditions and provides the benefit of allowing users to quickly identify problematic vibration events occur along the travel path. This teaches displaying a vibration point mark on the map indicating a point where vibration exceeds a predetermined threshold.) an operation reception process of receiving an operation of selecting the vibration point mark is displayed through the vibration point mark display process (Cella describes, ¶0359, 0361, that clicking, touching, or interacting with a visual element on a map allows a user to interact with the display and drill down into underlying data. This teaches an operation reception process of receiving an operation of selecting the vibration point mark.) the replay display process from a point upstream from a point indicated by the vibration point mark in response to the operation of selecting the vibration point mark is received through the operation reception process (Cella; ¶0431, describes playback controls including advance to time point, retreat to time point, and play. Kim’s recoded data (¶0502) includes time and position information. It would have been obvious to one of ordinary skill in the art to configure the replay display process to start from a point before the selected vibration point mark in response to the user’s selection because Cella teaches time-based navigation controls (¶0431), and starting replay before the vibration event provides the predictable benefit of allowing the user to observe conditions leading up to the vibration threshold. Upstream from a point on the travel path reads on a point earlier in time along the recorded path, by using Cella’s time navigation controls to retreat to a time point before the vibration event. This teaches performing the replay display process from a point upstream from the vibration point mark in response to selecting it.)
Regarding claim 9, Kim in view of Cella teaches the information display system according to claim 3, wherein: the vehicle state information comprises at least one of a travel speed of the transport vehicle, an acceleration state of the transport vehicle, an operation state of a traveler included in the transport vehicle, or a detection state of a sensor included in the transport vehicle (Kim; ¶0500, describes that the data provided to the EDR may include speed and/or acceleration. Speed reads on travel speed of the transport vehicle and acceleration reads on an acceleration state of the transport vehicle. This teaches that the vehicle state information comprises at least one of a travel speed of the vehicle or an acceleration state of the vehicle.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAN F KALHORI whose telephone number is (571)272-5475. The examiner can normally be reached Mon-Fri 8:30-5:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached at (571) 272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAN F KALHORI/Examiner, Art Unit 2618
/ZHENGXI LIU/Primary Examiner, Art Unit 2611